From python-3000-checkins at python.org Thu May 1 00:33:08 2008 From: python-3000-checkins at python.org (thomas.heller) Date: Thu, 1 May 2008 00:33:08 +0200 (CEST) Subject: [Python-3000-checkins] r62609 - python/branches/py3k-ctypes-pep3118 Message-ID: <20080430223308.D23B61E4005@bag.python.org> Author: thomas.heller Date: Thu May 1 00:33:08 2008 New Revision: 62609 Log: Remove this branch, it has been merged into py3k. Removed: python/branches/py3k-ctypes-pep3118/ From python-3000-checkins at python.org Thu May 1 00:36:24 2008 From: python-3000-checkins at python.org (thomas.heller) Date: Thu, 1 May 2008 00:36:24 +0200 (CEST) Subject: [Python-3000-checkins] r62610 - python/branches/py3k Message-ID: <20080430223624.587FF1E4005@bag.python.org> Author: thomas.heller Date: Thu May 1 00:36:24 2008 New Revision: 62610 Log: Removed merge tracking for "svnmerge" for svn+ssh://pythondev at svn.python.org/python/branches/py3k-ctypes-pep3118 Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Thu May 1 01:30:58 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Thu, 1 May 2008 01:30:58 +0200 (CEST) Subject: [Python-3000-checkins] r62611 - in python/branches/py3k: Lib/test/ieee754.txt Lib/test/test_math.py Modules/mathmodule.c Message-ID: <20080430233058.411141E4005@bag.python.org> Author: mark.dickinson Date: Thu May 1 01:30:57 2008 New Revision: 62611 Log: Make floating-point exception error messages slightly more verbose: in particular, the error message now allows one to distinguish between a ValueError arising from a singularity (e.g. log(0.)), which would usually produce +-infinity in non-stop mode, and a ValueError resulting from an invalid input (e.g. sqrt(-1.)), which would normally produce a NaN in non-stop mode. Modified: python/branches/py3k/Lib/test/ieee754.txt python/branches/py3k/Lib/test/test_math.py python/branches/py3k/Modules/mathmodule.c Modified: python/branches/py3k/Lib/test/ieee754.txt ============================================================================== --- python/branches/py3k/Lib/test/ieee754.txt (original) +++ python/branches/py3k/Lib/test/ieee754.txt Thu May 1 01:30:57 2008 @@ -127,31 +127,31 @@ >>> sin(INF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> sin(NINF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> sin(NAN) nan >>> cos(INF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> cos(NINF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> cos(NAN) nan >>> tan(INF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> tan(NINF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> tan(NAN) nan @@ -169,11 +169,11 @@ >>> asin(INF), asin(NINF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> acos(INF), acos(NINF) Traceback (most recent call last): ... -ValueError: math domain error +ValueError: math domain error (invalid argument) >>> equal(atan(INF), PI/2), equal(atan(NINF), -PI/2) (True, True) Modified: python/branches/py3k/Lib/test/test_math.py ============================================================================== --- python/branches/py3k/Lib/test/test_math.py (original) +++ python/branches/py3k/Lib/test/test_math.py Thu May 1 01:30:57 2008 @@ -741,9 +741,9 @@ func = getattr(math, fn) try: result = func(ar) - except ValueError: - message = ("Unexpected ValueError in " + - "test %s:%s(%r)\n" % (id, fn, ar)) + except ValueError as exc: + message = (("Unexpected ValueError: %s\n " + + "in test %s:%s(%r)\n") % (exc.args[0], id, fn, ar)) self.fail(message) self.ftest("%s:%s(%r)" % (id, fn, ar), result, er) Modified: python/branches/py3k/Modules/mathmodule.c ============================================================================== --- python/branches/py3k/Modules/mathmodule.c (original) +++ python/branches/py3k/Modules/mathmodule.c Thu May 1 01:30:57 2008 @@ -174,18 +174,21 @@ PyFPE_START_PROTECT("in math_1", return 0); r = (*func)(x); PyFPE_END_PROTECT(r); - if (Py_IS_NAN(r)) { - if (!Py_IS_NAN(x)) - errno = EDOM; - else - errno = 0; + if (Py_IS_NAN(r) && !Py_IS_NAN(x)) { + PyErr_SetString(PyExc_ValueError, + "math domain error (invalid argument)"); + return NULL; } - else if (Py_IS_INFINITY(r)) { - if (Py_IS_FINITE(x)) - errno = can_overflow ? ERANGE : EDOM; - else - errno = 0; + if (Py_IS_INFINITY(r) && Py_IS_FINITE(x)) { + if (can_overflow) + PyErr_SetString(PyExc_OverflowError, + "math range error (overflow)"); + else + PyErr_SetString(PyExc_ValueError, + "math domain error (singularity)"); + return NULL; } + /* on most machines, errno should be 0 at this point */ if (errno && is_error(r)) return NULL; else From python-3000-checkins at python.org Thu May 1 02:19:23 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Thu, 1 May 2008 02:19:23 +0200 (CEST) Subject: [Python-3000-checkins] r62612 - python/branches/py3k/Modules/mathmodule.c Message-ID: <20080501001923.EDEF71E4005@bag.python.org> Author: mark.dickinson Date: Thu May 1 02:19:23 2008 New Revision: 62612 Log: Whoops. errno should only be tested when the result is finite. Modified: python/branches/py3k/Modules/mathmodule.c Modified: python/branches/py3k/Modules/mathmodule.c ============================================================================== --- python/branches/py3k/Modules/mathmodule.c (original) +++ python/branches/py3k/Modules/mathmodule.c Thu May 1 02:19:23 2008 @@ -188,11 +188,11 @@ "math domain error (singularity)"); return NULL; } - /* on most machines, errno should be 0 at this point */ - if (errno && is_error(r)) + if (Py_IS_FINITE(r) && errno && is_error(r)) + /* this branch unnecessary on most platforms */ return NULL; - else - return (*from_double_func)(r); + + return (*from_double_func)(r); } /* From python-3000-checkins at python.org Thu May 1 20:06:51 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Thu, 1 May 2008 20:06:51 +0200 (CEST) Subject: [Python-3000-checkins] r62615 - python/branches/py3k/Doc/tutorial/controlflow.rst Message-ID: <20080501180651.0B94E1E4025@bag.python.org> Author: georg.brandl Date: Thu May 1 20:06:50 2008 New Revision: 62615 Log: Don't output floats in prime example. Modified: python/branches/py3k/Doc/tutorial/controlflow.rst Modified: python/branches/py3k/Doc/tutorial/controlflow.rst ============================================================================== --- python/branches/py3k/Doc/tutorial/controlflow.rst (original) +++ python/branches/py3k/Doc/tutorial/controlflow.rst Thu May 1 20:06:50 2008 @@ -166,7 +166,7 @@ >>> for n in range(2, 10): ... for x in range(2, n): ... if n % x == 0: - ... print(n, 'equals', x, '*', n/x) + ... print(n, 'equals', x, '*', n//x) ... break ... else: ... # loop fell through without finding a factor From python-3000-checkins at python.org Thu May 1 23:27:06 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Thu, 1 May 2008 23:27:06 +0200 (CEST) Subject: [Python-3000-checkins] r62620 - in python/branches/py3k: configure configure.in Message-ID: <20080501212706.08D331E400F@bag.python.org> Author: mark.dickinson Date: Thu May 1 23:27:05 2008 New Revision: 62620 Log: Remove temporary autoconf checks added in revision 62592 Modified: python/branches/py3k/configure python/branches/py3k/configure.in Modified: python/branches/py3k/configure ============================================================================== --- python/branches/py3k/configure (original) +++ python/branches/py3k/configure Thu May 1 23:27:05 2008 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 62515 . +# From configure.in Revision: 62592 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.61 for python 3.0. # @@ -21082,435 +21082,6 @@ LIBS_SAVE=$LIBS LIBS="$LIBS $LIBM" -# temporary checks to try to track down what's going wrong -# with test_math on Debian/alpha. These checks will be -# removed later. - -case $ac_sys_machine in -alpha*) - - { echo "$as_me:$LINENO: checking whether 9.88e-324 compares unequal to 0.0" >&5 -echo $ECHO_N "checking whether 9.88e-324 compares unequal to 0.0... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_subnormal_nonzero=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - int main() { - double x = 9.88e-324; - if (x != 0.0) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_subnormal_nonzero=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_subnormal_nonzero=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_subnormal_nonzero" >&5 -echo "${ECHO_T}$ac_cv_subnormal_nonzero" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) succeeds" >&5 -echo $ECHO_N "checking whether log(9.88e-324) succeeds... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_succeeds=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - exit(0); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_succeeds=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_succeeds=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_succeeds" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_succeeds" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) returns correct result" >&5 -echo $ECHO_N "checking whether log(9.88e-324) returns correct result... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_returns_correct_result=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if (-744. < x && x < -743.) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_returns_correct_result=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_returns_correct_result=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_returns_correct_result" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_returns_correct_result" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) sets errno" >&5 -echo $ECHO_N "checking whether log(9.88e-324) sets errno... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_sets_errno=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - #include - int main() { - double x = 9.88e-324; - errno = 0; - x = log(x); - if (errno != 0) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_sets_errno=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_sets_errno=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_sets_errno" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_sets_errno" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) sets errno = EDOM" >&5 -echo $ECHO_N "checking whether log(9.88e-324) sets errno = EDOM... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_sets_errno_to_EDOM=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - #include - int main() { - double x = 9.88e-324; - errno = 0; - x = log(x); - if (errno == EDOM) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_sets_errno_to_EDOM=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_sets_errno_to_EDOM=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_sets_errno_to_EDOM" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_sets_errno_to_EDOM" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) is infinite" >&5 -echo $ECHO_N "checking whether log(9.88e-324) is infinite... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_is_infinite=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if ((x > 1. || x < -1.) && x/2. == x) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_is_infinite=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_is_infinite=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_is_infinite" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_is_infinite" >&6; } - - { echo "$as_me:$LINENO: checking whether log(9.88e-324) is a nan" >&5 -echo $ECHO_N "checking whether log(9.88e-324) is a nan... $ECHO_C" >&6; } - if test "$cross_compiling" = yes; then - ac_cv_log_subnormal_is_nan=no -else - cat >conftest.$ac_ext <<_ACEOF -/* confdefs.h. */ -_ACEOF -cat confdefs.h >>conftest.$ac_ext -cat >>conftest.$ac_ext <<_ACEOF -/* end confdefs.h. */ - - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if (x != x) - exit(0); - else - exit(1); - } - -_ACEOF -rm -f conftest$ac_exeext -if { (ac_try="$ac_link" -case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_link") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); } && { ac_try='./conftest$ac_exeext' - { (case "(($ac_try" in - *\"* | *\`* | *\\*) ac_try_echo=\$ac_try;; - *) ac_try_echo=$ac_try;; -esac -eval "echo \"\$as_me:$LINENO: $ac_try_echo\"") >&5 - (eval "$ac_try") 2>&5 - ac_status=$? - echo "$as_me:$LINENO: \$? = $ac_status" >&5 - (exit $ac_status); }; }; then - ac_cv_log_subnormal_is_nan=yes -else - echo "$as_me: program exited with status $ac_status" >&5 -echo "$as_me: failed program was:" >&5 -sed 's/^/| /' conftest.$ac_ext >&5 - -( exit $ac_status ) -ac_cv_log_subnormal_is_nan=no -fi -rm -f core *.core core.conftest.* gmon.out bb.out conftest$ac_exeext conftest.$ac_objext conftest.$ac_ext -fi - - - { echo "$as_me:$LINENO: result: $ac_cv_log_subnormal_is_nan" >&5 -echo "${ECHO_T}$ac_cv_log_subnormal_is_nan" >&6; } -;; -esac - - for ac_func in hypot do as_ac_var=`echo "ac_cv_func_$ac_func" | $as_tr_sh` Modified: python/branches/py3k/configure.in ============================================================================== --- python/branches/py3k/configure.in (original) +++ python/branches/py3k/configure.in Thu May 1 23:27:05 2008 @@ -2980,141 +2980,6 @@ # ************************************ LIBS_SAVE=$LIBS LIBS="$LIBS $LIBM" - -# temporary checks to try to track down what's going wrong -# with test_math on Debian/alpha. These checks will be -# removed later. - -case $ac_sys_machine in -alpha*) - - AC_MSG_CHECKING(whether 9.88e-324 compares unequal to 0.0) - AC_TRY_RUN([ - #include - int main() { - double x = 9.88e-324; - if (x != 0.0) - exit(0); - else - exit(1); - } - ], - ac_cv_subnormal_nonzero=yes, - ac_cv_subnormal_nonzero=no, - ac_cv_subnormal_nonzero=no) - AC_MSG_RESULT($ac_cv_subnormal_nonzero) - - AC_MSG_CHECKING(whether log(9.88e-324) succeeds) - AC_TRY_RUN([ - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - exit(0); - } - ], - ac_cv_log_subnormal_succeeds=yes, - ac_cv_log_subnormal_succeeds=no, - ac_cv_log_subnormal_succeeds=no) - AC_MSG_RESULT($ac_cv_log_subnormal_succeeds) - - AC_MSG_CHECKING(whether log(9.88e-324) returns correct result) - AC_TRY_RUN([ - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if (-744. < x && x < -743.) - exit(0); - else - exit(1); - } - ], - ac_cv_log_subnormal_returns_correct_result=yes, - ac_cv_log_subnormal_returns_correct_result=no, - ac_cv_log_subnormal_returns_correct_result=no) - AC_MSG_RESULT($ac_cv_log_subnormal_returns_correct_result) - - AC_MSG_CHECKING(whether log(9.88e-324) sets errno) - AC_TRY_RUN([ - #include - #include - #include - int main() { - double x = 9.88e-324; - errno = 0; - x = log(x); - if (errno != 0) - exit(0); - else - exit(1); - } - ], - ac_cv_log_subnormal_sets_errno=yes, - ac_cv_log_subnormal_sets_errno=no, - ac_cv_log_subnormal_sets_errno=no) - AC_MSG_RESULT($ac_cv_log_subnormal_sets_errno) - - AC_MSG_CHECKING(whether log(9.88e-324) sets errno = EDOM) - AC_TRY_RUN([ - #include - #include - #include - int main() { - double x = 9.88e-324; - errno = 0; - x = log(x); - if (errno == EDOM) - exit(0); - else - exit(1); - } - ], - ac_cv_log_subnormal_sets_errno_to_EDOM=yes, - ac_cv_log_subnormal_sets_errno_to_EDOM=no, - ac_cv_log_subnormal_sets_errno_to_EDOM=no) - AC_MSG_RESULT($ac_cv_log_subnormal_sets_errno_to_EDOM) - - AC_MSG_CHECKING(whether log(9.88e-324) is infinite) - AC_TRY_RUN([ - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if ((x > 1. || x < -1.) && x/2. == x) - exit(0); - else - exit(1); - } - ], - ac_cv_log_subnormal_is_infinite=yes, - ac_cv_log_subnormal_is_infinite=no, - ac_cv_log_subnormal_is_infinite=no) - AC_MSG_RESULT($ac_cv_log_subnormal_is_infinite) - - AC_MSG_CHECKING(whether log(9.88e-324) is a nan) - AC_TRY_RUN([ - #include - #include - int main() { - double x = 9.88e-324; - x = log(x); - if (x != x) - exit(0); - else - exit(1); - } - ], - ac_cv_log_subnormal_is_nan=yes, - ac_cv_log_subnormal_is_nan=no, - ac_cv_log_subnormal_is_nan=no) - AC_MSG_RESULT($ac_cv_log_subnormal_is_nan) -;; -esac - AC_REPLACE_FUNCS(hypot) AC_CHECK_FUNCS(acosh asinh atanh copysign expm1 finite isinf isnan log1p) From python-3000-checkins at python.org Fri May 2 03:19:51 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Fri, 2 May 2008 03:19:51 +0200 (CEST) Subject: [Python-3000-checkins] r62624 - python/branches/py3k/Modules/mathmodule.c Message-ID: <20080502011951.140821E400F@bag.python.org> Author: mark.dickinson Date: Fri May 2 03:19:50 2008 New Revision: 62624 Log: One more attempt to track down Debian/alpha test_math failures: add diagnostic information in the ValueError message. This change is temporary, and will be reversed after the next run of the Debian/alpha buildbot. Modified: python/branches/py3k/Modules/mathmodule.c Modified: python/branches/py3k/Modules/mathmodule.c ============================================================================== --- python/branches/py3k/Modules/mathmodule.c (original) +++ python/branches/py3k/Modules/mathmodule.c Fri May 2 03:19:50 2008 @@ -167,6 +167,7 @@ int can_overflow) { double x, r; + char err_message[150]; x = PyFloat_AsDouble(arg); if (x == -1.0 && PyErr_Occurred()) return NULL; @@ -183,9 +184,16 @@ if (can_overflow) PyErr_SetString(PyExc_OverflowError, "math range error (overflow)"); - else - PyErr_SetString(PyExc_ValueError, - "math domain error (singularity)"); + else { + /* temporary code to include the inputs + and outputs to func in the error + message */ + sprintf(err_message, + "math domain error (singularity) " + "%.17g -> %.17g", + x, r); + PyErr_SetString(PyExc_ValueError, err_message); + } return NULL; } if (Py_IS_FINITE(r) && errno && is_error(r)) From python-3000-checkins at python.org Sat May 3 06:39:39 2008 From: python-3000-checkins at python.org (alexandre.vassalotti) Date: Sat, 3 May 2008 06:39:39 +0200 (CEST) Subject: [Python-3000-checkins] r62657 - in python/branches/py3k/Lib: base64.py email/base64mime.py test/test_base64.py urllib2.py Message-ID: <20080503043939.6F6BA1E4019@bag.python.org> Author: alexandre.vassalotti Date: Sat May 3 06:39:38 2008 New Revision: 62657 Log: Removed implicit convertions of str object to bytes from base64. This also exposed some bugs in urlib2 and email.base64mime, which I tried my best to fix. However, someone will probably have to double check. Modified: python/branches/py3k/Lib/base64.py python/branches/py3k/Lib/email/base64mime.py python/branches/py3k/Lib/test/test_base64.py python/branches/py3k/Lib/urllib2.py Modified: python/branches/py3k/Lib/base64.py ============================================================================== --- python/branches/py3k/Lib/base64.py (original) +++ python/branches/py3k/Lib/base64.py Sat May 3 06:39:38 2008 @@ -53,12 +53,13 @@ The encoded byte string is returned. """ if not isinstance(s, bytes_types): - s = bytes(s, "ascii") + raise TypeError("expected bytes, not %s" % s.__class__.__name__) # Strip off the trailing newline encoded = binascii.b2a_base64(s)[:-1] if altchars is not None: if not isinstance(altchars, bytes_types): - altchars = bytes(altchars, "ascii") + altchars = TypeError("expected bytes, not %s" + % altchars.__class__.__name__) assert len(altchars) == 2, repr(altchars) return _translate(encoded, {'+': altchars[0:1], '/': altchars[1:2]}) return encoded @@ -76,10 +77,11 @@ present in the string. """ if not isinstance(s, bytes_types): - s = bytes(s) + raise TypeError("expected bytes, not %s" % s.__class__.__name__) if altchars is not None: if not isinstance(altchars, bytes_types): - altchars = bytes(altchars, "ascii") + raise TypeError("expected bytes, not %s" + % altchars.__class__.__name__) assert len(altchars) == 2, repr(altchars) s = _translate(s, {chr(altchars[0]): b'+', chr(altchars[1]): b'/'}) return binascii.a2b_base64(s) @@ -148,7 +150,7 @@ s is the byte string to encode. The encoded byte string is returned. """ if not isinstance(s, bytes_types): - s = bytes(s) + raise TypeError("expected bytes, not %s" % s.__class__.__name__) quanta, leftover = divmod(len(s), 5) # Pad the last quantum with zero bits if necessary if leftover: @@ -205,16 +207,16 @@ characters present in the input. """ if not isinstance(s, bytes_types): - s = bytes(s) + raise TypeError("expected bytes, not %s" % s.__class__.__name__) quanta, leftover = divmod(len(s), 8) if leftover: raise binascii.Error('Incorrect padding') # Handle section 2.4 zero and one mapping. The flag map01 will be either # False, or the character to map the digit 1 (one) to. It should be # either L (el) or I (eye). - if map01: + if map01 is not None: if not isinstance(map01, bytes_types): - map01 = bytes(map01) + raise TypeError("expected bytes, not %s" % map01.__class__.__name__) assert len(map01) == 1, repr(map01) s = _translate(s, {b'0': b'O', b'1': map01}) if casefold: @@ -269,6 +271,8 @@ s is the byte string to encode. The encoded byte string is returned. """ + if not isinstance(s, bytes_types): + raise TypeError("expected bytes, not %s" % s.__class__.__name__) return binascii.hexlify(s).upper() @@ -284,7 +288,7 @@ present in the string. """ if not isinstance(s, bytes_types): - s = bytes(s) + raise TypeError("expected bytes, not %s" % s.__class__.__name__) if casefold: s = s.upper() if re.search('[^0-9A-F]', s): Modified: python/branches/py3k/Lib/email/base64mime.py ============================================================================== --- python/branches/py3k/Lib/email/base64mime.py (original) +++ python/branches/py3k/Lib/email/base64mime.py Sat May 3 06:39:38 2008 @@ -66,9 +66,10 @@ charset names the character set to use to encode the header. It defaults to iso-8859-1. Base64 encoding is defined in RFC 2045. """ - # Return empty headers unchanged if not header_bytes: - return str(header_bytes) + return "" + if isinstance(header_bytes, str): + header_bytes = header_bytes.encode(charset) encoded = b64encode(header_bytes).decode("ascii") return '=?%s?b?%s?=' % (charset, encoded) Modified: python/branches/py3k/Lib/test/test_base64.py ============================================================================== --- python/branches/py3k/Lib/test/test_base64.py (original) +++ python/branches/py3k/Lib/test/test_base64.py Sat May 3 06:39:38 2008 @@ -19,6 +19,7 @@ b"YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNE" b"RUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0\nNT" b"Y3ODkhQCMwXiYqKCk7Ojw+LC4gW117fQ==\n") + self.assertRaises(TypeError, base64.encodestring, "") def test_decodestring(self): eq = self.assertEqual @@ -33,6 +34,7 @@ b"ABCDEFGHIJKLMNOPQRSTUVWXYZ" b"0123456789!@#0^&*();:<>,. []{}") eq(base64.decodestring(b''), b'') + self.assertRaises(TypeError, base64.decodestring, "") def test_encode(self): eq = self.assertEqual @@ -54,7 +56,6 @@ base64.decode(infp, outfp) self.assertEqual(outfp.getvalue(), b'www.python.org') - class BaseXYTestCase(unittest.TestCase): def test_b64encode(self): @@ -73,7 +74,10 @@ b"RUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NT" b"Y3ODkhQCMwXiYqKCk7Ojw+LC4gW117fQ==") # Test with arbitrary alternative characters - eq(base64.b64encode(b'\xd3V\xbeo\xf7\x1d', altchars='*$'), b'01a*b$cd') + eq(base64.b64encode(b'\xd3V\xbeo\xf7\x1d', altchars=b'*$'), b'01a*b$cd') + # Check if passing a str object raises an error + self.assertRaises(TypeError, base64.b64encode, "") + self.assertRaises(TypeError, base64.b64encode, b"", altchars="") # Test standard alphabet eq(base64.standard_b64encode(b"www.python.org"), b"d3d3LnB5dGhvbi5vcmc=") eq(base64.standard_b64encode(b"a"), b"YQ==") @@ -86,8 +90,13 @@ b"YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXpBQkNE" b"RUZHSElKS0xNTk9QUVJTVFVWV1hZWjAxMjM0NT" b"Y3ODkhQCMwXiYqKCk7Ojw+LC4gW117fQ==") + # Check if passing a str object raises an error + self.assertRaises(TypeError, base64.standard_b64encode, "") + self.assertRaises(TypeError, base64.standard_b64encode, b"", altchars="") # Test with 'URL safe' alternative characters eq(base64.urlsafe_b64encode(b'\xd3V\xbeo\xf7\x1d'), b'01a-b_cd') + # Check if passing a str object raises an error + self.assertRaises(TypeError, base64.urlsafe_b64encode, "") def test_b64decode(self): eq = self.assertEqual @@ -104,7 +113,10 @@ b"0123456789!@#0^&*();:<>,. []{}") eq(base64.b64decode(b''), b'') # Test with arbitrary alternative characters - eq(base64.b64decode(b'01a*b$cd', altchars='*$'), b'\xd3V\xbeo\xf7\x1d') + eq(base64.b64decode(b'01a*b$cd', altchars=b'*$'), b'\xd3V\xbeo\xf7\x1d') + # Check if passing a str object raises an error + self.assertRaises(TypeError, base64.b64decode, "") + self.assertRaises(TypeError, base64.b64decode, b"", altchars="") # Test standard alphabet eq(base64.standard_b64decode(b"d3d3LnB5dGhvbi5vcmc="), b"www.python.org") eq(base64.standard_b64decode(b"YQ=="), b"a") @@ -117,8 +129,12 @@ b"abcdefghijklmnopqrstuvwxyz" b"ABCDEFGHIJKLMNOPQRSTUVWXYZ" b"0123456789!@#0^&*();:<>,. []{}") + # Check if passing a str object raises an error + self.assertRaises(TypeError, base64.standard_b64decode, "") + self.assertRaises(TypeError, base64.standard_b64decode, b"", altchars="") # Test with 'URL safe' alternative characters eq(base64.urlsafe_b64decode(b'01a-b_cd'), b'\xd3V\xbeo\xf7\x1d') + self.assertRaises(TypeError, base64.urlsafe_b64decode, "") def test_b64decode_error(self): self.assertRaises(binascii.Error, base64.b64decode, b'abc') @@ -132,6 +148,7 @@ eq(base64.b32encode(b'abc'), b'MFRGG===') eq(base64.b32encode(b'abcd'), b'MFRGGZA=') eq(base64.b32encode(b'abcde'), b'MFRGGZDF') + self.assertRaises(TypeError, base64.b32encode, "") def test_b32decode(self): eq = self.assertEqual @@ -142,6 +159,7 @@ eq(base64.b32decode(b'MFRGG==='), b'abc') eq(base64.b32decode(b'MFRGGZA='), b'abcd') eq(base64.b32decode(b'MFRGGZDF'), b'abcde') + self.assertRaises(TypeError, base64.b32decode, "") def test_b32decode_casefold(self): eq = self.assertEqual @@ -163,6 +181,7 @@ eq(base64.b32decode(b'MLO23456'), b'b\xdd\xad\xf3\xbe') eq(base64.b32decode(b'M1023456', map01=b'L'), b'b\xdd\xad\xf3\xbe') eq(base64.b32decode(b'M1023456', map01=b'I'), b'b\x1d\xad\xf3\xbe') + self.assertRaises(TypeError, base64.b32decode, b"", map01="") def test_b32decode_error(self): self.assertRaises(binascii.Error, base64.b32decode, b'abc') @@ -172,6 +191,7 @@ eq = self.assertEqual eq(base64.b16encode(b'\x01\x02\xab\xcd\xef'), b'0102ABCDEF') eq(base64.b16encode(b'\x00'), b'00') + self.assertRaises(TypeError, base64.b16encode, "") def test_b16decode(self): eq = self.assertEqual @@ -181,6 +201,7 @@ self.assertRaises(binascii.Error, base64.b16decode, b'0102abcdef') # Case fold eq(base64.b16decode(b'0102abcdef', True), b'\x01\x02\xab\xcd\xef') + self.assertRaises(TypeError, base64.b16decode, "") def test_ErrorHeritage(self): self.assert_(issubclass(binascii.Error, ValueError)) Modified: python/branches/py3k/Lib/urllib2.py ============================================================================== --- python/branches/py3k/Lib/urllib2.py (original) +++ python/branches/py3k/Lib/urllib2.py Sat May 3 06:39:38 2008 @@ -682,7 +682,7 @@ proxy_type = orig_type if user and password: user_pass = '%s:%s' % (unquote(user), unquote(password)) - creds = str(base64.b64encode(user_pass)).strip() + creds = base64.b64encode(user_pass.encode()).decode("ascii") req.add_header('Proxy-authorization', 'Basic ' + creds) hostport = unquote(hostport) req.set_proxy(hostport, proxy_type) @@ -808,7 +808,7 @@ user, pw = self.passwd.find_user_password(realm, host) if pw is not None: raw = "%s:%s" % (user, pw) - auth = 'Basic %s' % base64.b64encode(raw).strip().decode() + auth = "Basic " + base64.b64encode(raw.encode()).decode("ascii") if req.headers.get(self.auth_header, None) == auth: return None req.add_header(self.auth_header, auth) From python-3000-checkins at python.org Sat May 3 06:42:17 2008 From: python-3000-checkins at python.org (alexandre.vassalotti) Date: Sat, 3 May 2008 06:42:17 +0200 (CEST) Subject: [Python-3000-checkins] r62658 - python/branches/py3k/Objects/unicodeobject.c Message-ID: <20080503044217.1EB121E401A@bag.python.org> Author: alexandre.vassalotti Date: Sat May 3 06:42:16 2008 New Revision: 62658 Log: Fixed a small omission in the renaming of "unicode" to "str". Modified: python/branches/py3k/Objects/unicodeobject.c Modified: python/branches/py3k/Objects/unicodeobject.c ============================================================================== --- python/branches/py3k/Objects/unicodeobject.c (original) +++ python/branches/py3k/Objects/unicodeobject.c Sat May 3 06:42:16 2008 @@ -9062,7 +9062,7 @@ if (type != &PyUnicode_Type) return unicode_subtype_new(type, args, kwds); - if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:unicode", + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:str", kwlist, &x, &encoding, &errors)) return NULL; if (x == NULL) From python-3000-checkins at python.org Sat May 3 20:24:44 2008 From: python-3000-checkins at python.org (alexandre.vassalotti) Date: Sat, 3 May 2008 20:24:44 +0200 (CEST) Subject: [Python-3000-checkins] r62667 - in python/branches/py3k: Modules/datetimemodule.c Modules/parsermodule.c Modules/zipimport.c Objects/typeobject.c Python/import.c Python/structmember.c Message-ID: <20080503182444.555461E4015@bag.python.org> Author: alexandre.vassalotti Date: Sat May 3 20:24:43 2008 New Revision: 62667 Log: Issue #1950: Fixed misusage of PyUnicode_AsString(). Modified: python/branches/py3k/Modules/datetimemodule.c python/branches/py3k/Modules/parsermodule.c python/branches/py3k/Modules/zipimport.c python/branches/py3k/Objects/typeobject.c python/branches/py3k/Python/import.c python/branches/py3k/Python/structmember.c Modified: python/branches/py3k/Modules/datetimemodule.c ============================================================================== --- python/branches/py3k/Modules/datetimemodule.c (original) +++ python/branches/py3k/Modules/datetimemodule.c Sat May 3 20:24:43 2008 @@ -1217,10 +1217,9 @@ assert(object && format && timetuple); assert(PyUnicode_Check(format)); /* Convert the input format to a C string and size */ - pin = PyUnicode_AsString(format); + pin = PyUnicode_AsStringAndSize(format, &flen); if (!pin) return NULL; - flen = PyUnicode_GetSize(format); /* Give up if the year is before 1900. * Python strftime() plays games with the year, and different Modified: python/branches/py3k/Modules/parsermodule.c ============================================================================== --- python/branches/py3k/Modules/parsermodule.c (original) +++ python/branches/py3k/Modules/parsermodule.c Sat May 3 20:24:43 2008 @@ -717,11 +717,10 @@ Py_DECREF(o); } } - temp_str = PyUnicode_AsString(temp); - len = PyUnicode_GET_SIZE(temp) + 1; - strn = (char *)PyObject_MALLOC(len); + temp_str = PyUnicode_AsStringAndSize(temp, &len); + strn = (char *)PyObject_MALLOC(len + 1); if (strn != NULL) - (void) memcpy(strn, temp_str, len); + (void) memcpy(strn, temp_str, len + 1); Py_DECREF(temp); } else if (!ISNONTERMINAL(type)) { @@ -807,11 +806,10 @@ if (res && encoding) { Py_ssize_t len; const char *temp; - temp = PyUnicode_AsString(encoding); - len = PyUnicode_GET_SIZE(encoding) + 1; - res->n_str = (char *)PyObject_MALLOC(len); + temp = PyUnicode_AsStringAndSize(encoding, &len); + res->n_str = (char *)PyObject_MALLOC(len + 1); if (res->n_str != NULL && temp != NULL) - (void) memcpy(res->n_str, temp, len); + (void) memcpy(res->n_str, temp, len + 1); Py_DECREF(encoding); Py_DECREF(tuple); } Modified: python/branches/py3k/Modules/zipimport.c ============================================================================== --- python/branches/py3k/Modules/zipimport.c (original) +++ python/branches/py3k/Modules/zipimport.c Sat May 3 20:24:43 2008 @@ -61,16 +61,14 @@ zipimporter_init(ZipImporter *self, PyObject *args, PyObject *kwds) { char *path, *p, *prefix, buf[MAXPATHLEN+2]; - size_t len; + Py_ssize_t len; if (!_PyArg_NoKeywords("zipimporter()", kwds)) return -1; - if (!PyArg_ParseTuple(args, "s:zipimporter", - &path)) + if (!PyArg_ParseTuple(args, "s#:zipimporter", &path, &len)) return -1; - len = strlen(path); if (len == 0) { PyErr_SetString(ZipImportError, "archive path is empty"); return -1; @@ -329,7 +327,7 @@ fullpath = PyUnicode_FromFormat("%s%c%s%s", PyUnicode_AsString(self->archive), SEP, - *prefix ? prefix : "", + prefix ? prefix : "", subname); if (fullpath == NULL) goto error; @@ -388,6 +386,7 @@ #endif PyObject *toc_entry; Py_ssize_t len; + char *archive_str; if (!PyArg_ParseTuple(args, "s:zipimporter.get_data", &path)) return NULL; @@ -404,9 +403,9 @@ } path = buf; #endif - len = PyUnicode_GET_SIZE(self->archive); + archive_str = PyUnicode_AsStringAndSize(self->archive, &len); if ((size_t)len < strlen(path) && - strncmp(path, PyUnicode_AsString(self->archive), len) == 0 && + strncmp(path, archive_str, len) == 0 && path[len] == SEP) { path = path + len + 1; } @@ -416,7 +415,7 @@ PyErr_SetFromErrnoWithFilename(PyExc_IOError, path); return NULL; } - return get_data(PyUnicode_AsString(self->archive), toc_entry); + return get_data(archive_str, toc_entry); } static PyObject * Modified: python/branches/py3k/Objects/typeobject.c ============================================================================== --- python/branches/py3k/Objects/typeobject.c (original) +++ python/branches/py3k/Objects/typeobject.c Sat May 3 20:24:43 2008 @@ -1255,7 +1255,7 @@ if (PyList_GET_ITEM(list, j) == o) { o = class_name(o); PyErr_Format(PyExc_TypeError, - "duplicate base class %s", + "duplicate base class %.400s", o ? PyUnicode_AsString(o) : "?"); Py_XDECREF(o); return -1; @@ -2133,20 +2133,27 @@ { PyObject *doc = PyDict_GetItemString(dict, "__doc__"); if (doc != NULL && PyUnicode_Check(doc)) { - size_t n; + Py_ssize_t len; + char *doc_str; char *tp_doc; - const char *str = PyUnicode_AsString(doc); - if (str == NULL) { + + doc_str = PyUnicode_AsStringAndSize(doc, &len); + if (doc_str == NULL) { + Py_DECREF(type); + return NULL; + } + if ((Py_ssize_t)strlen(doc_str) != len) { + PyErr_SetString(PyExc_TypeError, + "__doc__ contains null-bytes"); Py_DECREF(type); return NULL; } - n = strlen(str); - tp_doc = (char *)PyObject_MALLOC(n+1); + tp_doc = (char *)PyObject_MALLOC(len + 1); if (tp_doc == NULL) { Py_DECREF(type); return NULL; } - memcpy(tp_doc, str, n+1); + memcpy(tp_doc, doc_str, len + 1); type->tp_doc = tp_doc; } } Modified: python/branches/py3k/Python/import.c ============================================================================== --- python/branches/py3k/Python/import.c (original) +++ python/branches/py3k/Python/import.c Sat May 3 20:24:43 2008 @@ -2131,13 +2131,15 @@ if ((pkgname != NULL) && (pkgname != Py_None)) { /* __package__ is set, so use it */ + char *pkgname_str; Py_ssize_t len; + if (!PyUnicode_Check(pkgname)) { PyErr_SetString(PyExc_ValueError, "__package__ set to non-string"); return NULL; } - len = PyUnicode_GET_SIZE(pkgname); + pkgname_str = PyUnicode_AsStringAndSize(pkgname, &len); if (len == 0) { if (level > 0) { PyErr_SetString(PyExc_ValueError, @@ -2151,7 +2153,7 @@ "Package name too long"); return NULL; } - strcpy(buf, PyUnicode_AsString(pkgname)); + strcpy(buf, pkgname_str); } else { /* __package__ not set, so figure it out and set it */ modname = PyDict_GetItem(globals, namestr); @@ -2161,14 +2163,17 @@ modpath = PyDict_GetItem(globals, pathstr); if (modpath != NULL) { /* __path__ is set, so modname is already the package name */ - Py_ssize_t len = PyUnicode_GET_SIZE(modname); + char *modname_str; + Py_ssize_t len; int error; + + modname_str = PyUnicode_AsStringAndSize(modname, &len); if (len > MAXPATHLEN) { PyErr_SetString(PyExc_ValueError, "Module name too long"); return NULL; } - strcpy(buf, PyUnicode_AsString(modname)); + strcpy(buf, modname_str); error = PyDict_SetItem(globals, pkgstr, modname); if (error) { PyErr_SetString(PyExc_ValueError, Modified: python/branches/py3k/Python/structmember.c ============================================================================== --- python/branches/py3k/Python/structmember.c (original) +++ python/branches/py3k/Python/structmember.c Sat May 3 20:24:43 2008 @@ -239,15 +239,22 @@ *(PyObject **)addr = v; Py_XDECREF(oldv); break; - case T_CHAR: - if (PyUnicode_Check(v) && PyUnicode_GetSize(v) == 1) { - *(char*)addr = PyUnicode_AsString(v)[0]; + case T_CHAR: { + char *string; + Py_ssize_t len; + + if (!PyUnicode_Check(v)) { + PyErr_BadArgument(); + return -1; } - else { + string = PyUnicode_AsStringAndSize(v, &len); + if (len != 1) { PyErr_BadArgument(); return -1; } + *(char*)addr = string[0]; break; + } #ifdef HAVE_LONG_LONG case T_LONGLONG:{ PY_LONG_LONG value; From python-3000-checkins at python.org Sat May 3 20:25:45 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Sat, 3 May 2008 20:25:45 +0200 (CEST) Subject: [Python-3000-checkins] r62668 - python/branches/py3k Message-ID: <20080503182545.BF1B51E4015@bag.python.org> Author: mark.dickinson Date: Sat May 3 20:25:45 2008 New Revision: 62668 Log: block r62666 Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Sat May 3 21:59:07 2008 From: python-3000-checkins at python.org (mark.dickinson) Date: Sat, 3 May 2008 21:59:07 +0200 (CEST) Subject: [Python-3000-checkins] r62669 - python/branches/py3k/Lib/decimal.py Message-ID: <20080503195907.69FB11E4020@bag.python.org> Author: mark.dickinson Date: Sat May 3 21:59:07 2008 New Revision: 62669 Log: Remove broken __round__, __ceil__ and __floor__ from Decimal Modified: python/branches/py3k/Lib/decimal.py Modified: python/branches/py3k/Lib/decimal.py ============================================================================== --- python/branches/py3k/Lib/decimal.py (original) +++ python/branches/py3k/Lib/decimal.py Sat May 3 21:59:07 2008 @@ -1649,9 +1649,6 @@ else: return -1 - def __round__(self): - return self._round_down(0) - def _round_up(self, prec): """Rounds away from 0.""" return -self._round_down(prec) @@ -1687,9 +1684,6 @@ else: return -self._round_down(prec) - def __ceil__(self): - return self._round_ceiling(0) - def _round_floor(self, prec): """Rounds down (not towards 0 if negative)""" if not self._sign: @@ -1697,9 +1691,6 @@ else: return -self._round_down(prec) - def __floor__(self): - return self._round_floor(0) - def _round_05up(self, prec): """Round down unless digit prec-1 is 0 or 5.""" if prec and self._int[prec-1] not in '05': From python-3000-checkins at python.org Sat May 3 22:52:19 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Sat, 3 May 2008 22:52:19 +0200 (CEST) Subject: [Python-3000-checkins] r62679 - in python/branches/py3k: Doc/conf.py Doc/tools/sphinxext/layout.html Doc/tools/sphinxext/opensearch.xml Doc/tools/sphinxext/static Message-ID: <20080503205219.21DFC1E4013@bag.python.org> Author: georg.brandl Date: Sat May 3 22:52:18 2008 New Revision: 62679 Log: Merge OpenSearch-related changes. Added: python/branches/py3k/Doc/tools/sphinxext/layout.html - copied unchanged from r62540, /python/trunk/Doc/tools/sphinxext/layout.html python/branches/py3k/Doc/tools/sphinxext/opensearch.xml - copied unchanged from r62540, /python/trunk/Doc/tools/sphinxext/opensearch.xml python/branches/py3k/Doc/tools/sphinxext/static/ - copied from r62540, /python/trunk/Doc/tools/sphinxext/static/ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Doc/conf.py Modified: python/branches/py3k/Doc/conf.py ============================================================================== --- python/branches/py3k/Doc/conf.py (original) +++ python/branches/py3k/Doc/conf.py Sat May 3 22:52:18 2008 @@ -86,6 +86,12 @@ 'index': 'indexcontent.html', } +# Output an OpenSearch description file. +html_use_opensearch = 'http://docs.python.org/dev/3.0' + +# Additional static files. +html_static_path = ['tools/sphinxext/static'] + # Output file base name for HTML help builder. htmlhelp_basename = 'python' + release.replace('.', '') From python-3000-checkins at python.org Mon May 5 00:42:05 2008 From: python-3000-checkins at python.org (christian.heimes) Date: Mon, 5 May 2008 00:42:05 +0200 (CEST) Subject: [Python-3000-checkins] r62716 - in python/branches/py3k: Doc/about.rst Doc/bugs.rst Doc/c-api/long.rst Doc/c-api/number.rst Doc/distutils/builtdist.rst Doc/library/carbon.rst Doc/library/ctypes.rst Doc/library/getpass.rst Doc/library/io.rst Doc/library/optparse.rst Doc/library/pkgutil.rst Doc/library/pyclbr.rst Doc/library/robotparser.rst Doc/library/simplehttpserver.rst Doc/library/socket.rst Doc/library/sqlite3.rst Doc/library/subprocess.rst Doc/library/sys.rst Doc/library/tempfile.rst Doc/library/unittest.rst Doc/library/xmlrpclib.rst Doc/reference/expressions.rst Doc/tools/sphinxext/indexcontent.html Doc/using/cmdline.rst Doc/whatsnew/2.6.rst Lib/calendar.py Lib/contextlib.py Lib/ctypes/test/test_frombuffer.py Lib/decimal.py Lib/distutils/command/bdist_wininst.py Lib/distutils/command/wininst-6.0.exe Lib/distutils/command/wininst-7.1.exe Lib/distutils/command/wininst-9.0-amd64.exe Lib/distutils/command/wininst-9.0.exe Lib/distutils/msvc9compiler.py Lib/getpass.py Lib/idlelib/AutoComplete.py Lib/idlelib/AutoCompleteWindow.py Lib/idlelib/EditorWindow.py Lib/idlelib/PyShell.py Lib/idlelib/configHandler.py Lib/lib2to3 Lib/lib2to3/refactor.py Lib/lib2to3/tests/test_fixers.py Lib/plat-mac/Carbon/AppleEvents.py Lib/plat-mac/macerrors.py Lib/plat-mac/terminalcommand.py Lib/pyclbr.py Lib/robotparser.py Lib/sqlite3/test/factory.py Lib/sqlite3/test/types.py Lib/tempfile.py Lib/test/crashers/mutation_inside_cyclegc.py Lib/test/test_builtin.py Lib/test/test_cmd_line_script.py Lib/test/test_contextlib.py Lib/test/test_decimal.py Lib/test/test_float.py Lib/test/test_frozen.py Lib/test/test_import.py Lib/test/test_int.py Lib/test/test_io.py Lib/test/test_list.py Lib/test/test_long.py Lib/test/test_sundry.py Lib/test/test_support.py Lib/test/test_textwrap.py Lib/test/test_trace.py Lib/test/test_traceback.py Lib/test/test_tuple.py Lib/test/test_urllib2.py Lib/test/test_warnings.py Lib/test/test_with.py Lib/test/test_xmlrpc.py Lib/textwrap.py Lib/urllib2.py Lib/weakref.py Mac/BuildScript/build-installer.py Mac/BuildScript/resources/ReadMe.txt Mac/IDLE/Makefile.in Mac/Makefile.in Mac/PythonLauncher/Makefile.in Mac/PythonLauncher/MyDocument.m Mac/PythonLauncher/doscript.m Mac/Resources/app/Info.plist Mac/Resources/app/Info.plist.in Mac/Resources/app/Resources/English.lproj/InfoPlist.strings Mac/Resources/framework/English.lproj Mac/Resources/framework/Info.plist Mac/Resources/framework/Info.plist.in Mac/Resources/framework/version.plist Makefile.pre.in Misc/developers.txt Modules/_ctypes/_ctypes.c Modules/_sqlite/cursor.c Modules/_sqlite/row.c Modules/_sqlite/statement.c Modules/_testcapimodule.c Modules/mathmodule.c PC/bdist_wininst/install.c PC/bdist_wininst/wininst-7.1.vcproj PC/bdist_wininst/wininst.dsp Parser/tokenizer.c Python/_warnings.c Python/ast.c Python/traceback.c Tools/pybench/pybench.py Tools/pybench/systimes.py configure configure.in pyconfig.h.in Message-ID: <20080504224205.96B241E4004@bag.python.org> Author: christian.heimes Date: Mon May 5 00:42:01 2008 New Revision: 62716 Log: Merged revisions 62425-62429,62434-62436,62441,62444,62446-62448,62450-62455,62463,62465-62466,62469,62474,62476-62478,62480,62485,62492,62497-62498,62500,62507,62513-62514,62516,62521,62531,62535,62545-62546,62548-62551,62553-62559,62569,62574,62577,62593,62595,62604-62606,62608,62616,62626-62627,62636,62638,62644-62645,62647-62648,62651-62653,62656,62661,62663,62680,62686-62687,62696,62699-62703,62711 via svnmerge from svn+ssh://pythondev at svn.python.org/python/trunk ................ r62425 | andrew.kuchling | 2008-04-21 03:45:57 +0200 (Mon, 21 Apr 2008) | 1 line Comment typo ................ r62426 | mark.dickinson | 2008-04-21 03:55:50 +0200 (Mon, 21 Apr 2008) | 2 lines Silence 'r may be used uninitialized' compiler warning. ................ r62427 | andrew.kuchling | 2008-04-21 04:08:00 +0200 (Mon, 21 Apr 2008) | 1 line Markup fix ................ r62428 | andrew.kuchling | 2008-04-21 04:08:13 +0200 (Mon, 21 Apr 2008) | 1 line Wording changes ................ r62429 | andrew.kuchling | 2008-04-21 04:14:24 +0200 (Mon, 21 Apr 2008) | 1 line Add various items ................ r62434 | thomas.heller | 2008-04-21 15:46:55 +0200 (Mon, 21 Apr 2008) | 1 line Fix typo. ................ r62435 | david.goodger | 2008-04-21 16:40:22 +0200 (Mon, 21 Apr 2008) | 1 line corrections ("reStructuredText" is one word) ................ r62436 | david.goodger | 2008-04-21 16:43:33 +0200 (Mon, 21 Apr 2008) | 1 line capitalization ................ r62441 | gregory.p.smith | 2008-04-21 19:46:40 +0200 (Mon, 21 Apr 2008) | 2 lines explicitly flush after the ... since there wasn't a newline ................ r62444 | jeroen.ruigrok | 2008-04-21 22:15:39 +0200 (Mon, 21 Apr 2008) | 2 lines Windows x64 also falls under VER_PLATFORM_WIN32_NT. ................ r62446 | gregory.p.smith | 2008-04-21 23:31:08 +0200 (Mon, 21 Apr 2008) | 3 lines If sys.stdin is not a tty, fall back to default_getpass after printing a warning instead of failing with a termios.error. ................ r62447 | mark.dickinson | 2008-04-22 00:32:24 +0200 (Tue, 22 Apr 2008) | 8 lines test_math and test_cmath are failing on the FreeBSD 6.2 trunk buildbot, apparently because tanh(-0.) loses the sign of zero on that platform. If true, this is a bug in FreeBSD. Added a configure test to verify this. I still need to figure out how best to deal with this failure. ................ r62448 | amaury.forgeotdarc | 2008-04-22 00:35:30 +0200 (Tue, 22 Apr 2008) | 7 lines Issue 2665: On Windows, sys.stderr does not contain a valid file when running without a console. It seems to work, but will fail at the first flush. This causes IDLE to crash when too many warnings are printed. Will backport. ................ r62450 | benjamin.peterson | 2008-04-22 00:57:00 +0200 (Tue, 22 Apr 2008) | 2 lines Fix Sphinx warnings ................ r62451 | mark.dickinson | 2008-04-22 02:54:27 +0200 (Tue, 22 Apr 2008) | 3 lines Make configure test for tanh(-0.) == -0. committed in r62447 actually work. (The test wasn't properly linked with libm. Sigh.) ................ r62452 | benjamin.peterson | 2008-04-22 04:16:03 +0200 (Tue, 22 Apr 2008) | 2 lines Various io doc updates ................ r62453 | neal.norwitz | 2008-04-22 07:07:47 +0200 (Tue, 22 Apr 2008) | 1 line Add Thomas Lee ................ r62454 | gregory.p.smith | 2008-04-22 10:08:41 +0200 (Tue, 22 Apr 2008) | 8 lines Major improvements: * Default to using /dev/tty for the password prompt and input before falling back to sys.stdin and sys.stderr. * Use sys.stderr instead of sys.stdout. * print the 'password may be echoed' warning to stream used to display the prompt rather than always sys.stderr. * warn() with GetPassWarning when input may be echoed. ................ r62455 | gregory.p.smith | 2008-04-22 10:11:33 +0200 (Tue, 22 Apr 2008) | 2 lines update the getpass entry ................ r62463 | amaury.forgeotdarc | 2008-04-22 23:14:41 +0200 (Tue, 22 Apr 2008) | 5 lines Issue #2670: urllib2.build_opener() failed when two handlers derive the same default base class. Will backport. ................ r62465 | skip.montanaro | 2008-04-23 00:45:09 +0200 (Wed, 23 Apr 2008) | 3 lines Factor in documentation changes from issue 1753732. ................ r62466 | gregory.p.smith | 2008-04-23 03:06:42 +0200 (Wed, 23 Apr 2008) | 2 lines syntax fixup ................ r62469 | benjamin.peterson | 2008-04-23 22:38:06 +0200 (Wed, 23 Apr 2008) | 2 lines #2673 Fix example typo in optparse docs ................ r62474 | martin.v.loewis | 2008-04-24 11:50:50 +0200 (Thu, 24 Apr 2008) | 2 lines Add Guilherme Polo. ................ r62476 | martin.v.loewis | 2008-04-24 15:16:36 +0200 (Thu, 24 Apr 2008) | 3 lines Remove Py_Refcnt, Py_Type, Py_Size, as they were added only for backwards compatibility, yet 2.5 did not have them at all. ................ r62477 | martin.v.loewis | 2008-04-24 15:17:24 +0200 (Thu, 24 Apr 2008) | 2 lines Fix typo. ................ r62478 | martin.v.loewis | 2008-04-24 15:18:03 +0200 (Thu, 24 Apr 2008) | 2 lines Add Jesus Cea. ................ r62480 | amaury.forgeotdarc | 2008-04-24 20:07:05 +0200 (Thu, 24 Apr 2008) | 4 lines Issue2681: the literal 0o8 was wrongly accepted, and evaluated as float(0.0). This happened only when 8 is the first digit. Credits go to Lukas Meuser. ................ r62485 | amaury.forgeotdarc | 2008-04-24 22:10:26 +0200 (Thu, 24 Apr 2008) | 5 lines Disable gc when running test_trace, or we may record the __del__ of collected objects. See http://mail.python.org/pipermail/python-checkins/2008-April/068633.html the extra events perfectly match several calls to socket._fileobject.__del__() ................ r62492 | neal.norwitz | 2008-04-25 05:40:17 +0200 (Fri, 25 Apr 2008) | 1 line Fix typo (now -> no) ................ r62497 | armin.rigo | 2008-04-25 11:35:18 +0200 (Fri, 25 Apr 2008) | 2 lines A new crasher. ................ r62498 | thomas.heller | 2008-04-25 17:44:16 +0200 (Fri, 25 Apr 2008) | 1 line Add from_buffer and from_buffer_copy class methods to ctypes types. ................ r62500 | mark.dickinson | 2008-04-25 18:59:09 +0200 (Fri, 25 Apr 2008) | 3 lines Issue 2635: fix bug in the fix_sentence_endings option to textwrap.fill. ................ r62507 | benjamin.peterson | 2008-04-25 23:43:56 +0200 (Fri, 25 Apr 2008) | 2 lines Allow test_import to work when it is invoked directly ................ r62513 | georg.brandl | 2008-04-26 20:31:07 +0200 (Sat, 26 Apr 2008) | 2 lines #2691: document PyLong (s)size_t APIs, patch by Alexander Belopolsky. ................ r62514 | georg.brandl | 2008-04-26 20:32:17 +0200 (Sat, 26 Apr 2008) | 2 lines Add missing return type to dealloc. ................ r62516 | alexandre.vassalotti | 2008-04-27 02:52:24 +0200 (Sun, 27 Apr 2008) | 2 lines Fixed URL of PEP 205 in weakref's module docstring. ................ r62521 | georg.brandl | 2008-04-27 11:39:59 +0200 (Sun, 27 Apr 2008) | 2 lines #2677: add note that not all functions may accept keyword args. ................ r62531 | georg.brandl | 2008-04-27 19:38:55 +0200 (Sun, 27 Apr 2008) | 2 lines Use correct XHTML tags. ................ r62535 | benjamin.peterson | 2008-04-27 20:14:39 +0200 (Sun, 27 Apr 2008) | 2 lines #2700 Document PyNumber_ToBase ................ r62545 | skip.montanaro | 2008-04-27 22:53:57 +0200 (Sun, 27 Apr 2008) | 1 line minor wording changes, rewrap a few lines ................ r62546 | kurt.kaiser | 2008-04-27 23:07:41 +0200 (Sun, 27 Apr 2008) | 7 lines Home / Control-A toggles between left margin and end of leading white space. Patch 1196903 Jeff Shute. M idlelib/PyShell.py M idlelib/EditorWindow.py M idlelib/NEWS.txt ................ r62548 | kurt.kaiser | 2008-04-27 23:38:05 +0200 (Sun, 27 Apr 2008) | 2 lines Improved AutoCompleteWindow logic. Patch 2062 Tal Einat. ................ r62549 | kurt.kaiser | 2008-04-27 23:52:19 +0200 (Sun, 27 Apr 2008) | 4 lines Autocompletion of filenames now support alternate separators, e.g. the '/' char on Windows. Patch 2061 Tal Einat. ................ r62550 | skip.montanaro | 2008-04-28 00:49:56 +0200 (Mon, 28 Apr 2008) | 6 lines A few small changes: * The only exception we should catch when trying to import cStringIO is an ImportError. * Delete the function signatures embedded in the mk*temp docstrings. * The tempdir global variable was initialized twice. ................ r62551 | skip.montanaro | 2008-04-28 00:52:02 +0200 (Mon, 28 Apr 2008) | 4 lines Wrap some long paragraphs and include the default values for optional function parameters. ................ r62553 | skip.montanaro | 2008-04-28 04:57:23 +0200 (Mon, 28 Apr 2008) | 7 lines Minor cleanups: * Avoid creating unused local variables where we can. Where we can't prefix the unused variables with '_'. * Avoid shadowing builtins where it won't change the external interface of a function. * Use None as default path arg to readmodule and readmodule_ex. ................ r62554 | skip.montanaro | 2008-04-28 04:59:45 +0200 (Mon, 28 Apr 2008) | 6 lines Correct documentation to match implementation: "Class" instead of "class_descriptor", "Function" instead of "function_descriptor". Note default path value for readmodule*. Wrap some long paragraphs. Don't mention 'inpackage' which isn't part of the public API. ................ r62555 | brett.cannon | 2008-04-28 05:23:50 +0200 (Mon, 28 Apr 2008) | 5 lines Fix a bug introduced by the warnings rewrite where tracebacks were being improperly indented. Closes issue #2699. ................ r62556 | skip.montanaro | 2008-04-28 05:25:37 +0200 (Mon, 28 Apr 2008) | 2 lines Wrap some long lines. ................ r62557 | skip.montanaro | 2008-04-28 05:27:53 +0200 (Mon, 28 Apr 2008) | 6 lines Get rid of _test(), _main(), _debug() and _check(). Tests are no longer needed (better set available in Lib/test/test_robotparser.py). Clean up a few PEP 8 nits (compound statements on a single line, whitespace around operators). ................ r62558 | brett.cannon | 2008-04-28 06:50:06 +0200 (Mon, 28 Apr 2008) | 3 lines Rename the test_traceback_print() function to traceback_print() to prevent test_capi from automatically calling the function. ................ r62559 | georg.brandl | 2008-04-28 07:16:30 +0200 (Mon, 28 Apr 2008) | 2 lines Fix markup. ................ r62569 | amaury.forgeotdarc | 2008-04-28 23:07:06 +0200 (Mon, 28 Apr 2008) | 5 lines test_sundry performs minimal tests (a simple import...) on modules that are not tested otherwise. Some of them now have tests and can be removed. Only 70 to go... ................ r62574 | andrew.kuchling | 2008-04-29 04:03:54 +0200 (Tue, 29 Apr 2008) | 1 line Strip down SSL docs; I'm not managing to get test programs working, so I'll just give a minimal description ................ r62577 | martin.v.loewis | 2008-04-29 08:10:53 +0200 (Tue, 29 Apr 2008) | 2 lines Add Rodrigo and Heiko. ................ r62593 | nick.coghlan | 2008-04-30 16:23:36 +0200 (Wed, 30 Apr 2008) | 1 line Update command line usage documentation to reflect 2.6 changes (also includes some minor cleanups). Addresses TODO list issue 2258 ................ r62595 | andrew.kuchling | 2008-04-30 18:19:55 +0200 (Wed, 30 Apr 2008) | 1 line Typo fix ................ r62604 | benjamin.peterson | 2008-04-30 23:03:58 +0200 (Wed, 30 Apr 2008) | 2 lines make test_support's captured_output a bit more robust when exceptions happen ................ r62605 | georg.brandl | 2008-04-30 23:08:42 +0200 (Wed, 30 Apr 2008) | 2 lines #1748: use functools.wraps instead of rolling own metadata update. ................ r62606 | benjamin.peterson | 2008-04-30 23:25:55 +0200 (Wed, 30 Apr 2008) | 2 lines Remove some from __future__ import with_statements ................ r62608 | benjamin.peterson | 2008-05-01 00:03:36 +0200 (Thu, 01 May 2008) | 2 lines Fix typo in whatsnew ................ r62616 | georg.brandl | 2008-05-01 20:24:32 +0200 (Thu, 01 May 2008) | 2 lines Fix synopsis. ................ r62626 | brett.cannon | 2008-05-02 04:25:09 +0200 (Fri, 02 May 2008) | 6 lines Fix a backwards-compatibility mistake where a new optional argument for warnings.showwarning() was being used. This broke pre-existing replacements for the function since they didn't support the extra argument. Closes issue 2705. ................ r62627 | gregory.p.smith | 2008-05-02 09:26:52 +0200 (Fri, 02 May 2008) | 20 lines This should fix issue2632. A long description of the two competing problems is in the bug report (one old, one recently introduced trying to fix the old one). In short: buffer data during socket._fileobject.read() and readlines() within a cStringIO object instead of a [] of str()s returned from the recv() call. This prevents excessive memory use due to the size parameter being passed to recv() being grossly larger than the actual size of the data returned *and* prevents excessive cpu usage due to looping in python calling recv() with a very tiny size value if min() is used as the previous memory-use bug "fix" did. It also documents what the socket._fileobject._rbufsize member is actually used for. This is a candidate for back porting to 2.5. ................ r62636 | mark.hammond | 2008-05-02 14:48:15 +0200 (Fri, 02 May 2008) | 2 lines #2581: Vista UAC/elevation support for bdist_wininst ................ r62638 | facundo.batista | 2008-05-02 19:39:00 +0200 (Fri, 02 May 2008) | 3 lines Fixed some test structures. Thanks Mark Dickinson. ................ r62644 | ronald.oussoren | 2008-05-02 21:45:11 +0200 (Fri, 02 May 2008) | 7 lines Fix for issue #2573: Can't change the framework name on OS X builds This introduces a new configure option: --with-framework-name=NAME (defaulting to 'Python'). This allows you to install several copies of the Python framework with different names (such as a normal build and a debug build). ................ r62645 | ronald.oussoren | 2008-05-02 21:58:56 +0200 (Fri, 02 May 2008) | 2 lines Finish fix for issue2573, previous patch was incomplete. ................ r62647 | martin.v.loewis | 2008-05-02 23:30:20 +0200 (Fri, 02 May 2008) | 13 lines Merged revisions 62263-62646 via svnmerge from svn+ssh://pythondev at svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r62470 | david.wolever | 2008-04-24 02:11:07 +0200 (Do, 24 Apr 2008) | 3 lines Fixed up and applied the patch for #2431 -- speeding up 2to3 with a lookup table. ........ r62646 | martin.v.loewis | 2008-05-02 23:29:27 +0200 (Fr, 02 Mai 2008) | 2 lines Fix whitespace. ........ ................ r62648 | ronald.oussoren | 2008-05-02 23:42:35 +0200 (Fri, 02 May 2008) | 4 lines Fix for #1905: PythonLauncher not working correctly on OSX 10.5/Leopard This fixes both Python Launchar and the terminalcommand module. ................ r62651 | ronald.oussoren | 2008-05-02 23:54:56 +0200 (Fri, 02 May 2008) | 2 lines Fix for issue #2520 (cannot import macerrors) ................ r62652 | benjamin.peterson | 2008-05-03 00:12:58 +0200 (Sat, 03 May 2008) | 2 lines capitalization nit for reStructuredText ................ r62653 | brett.cannon | 2008-05-03 03:02:41 +0200 (Sat, 03 May 2008) | 2 lines Fix some indentation errors. ................ r62656 | brett.cannon | 2008-05-03 05:19:39 +0200 (Sat, 03 May 2008) | 6 lines Fix the C implementation of 'warnings' to infer the filename of the module that raised an exception properly when __file__ is not set, __name__ == '__main__', and sys.argv[0] is a false value. Closes issue2743. ................ r62661 | amaury.forgeotdarc | 2008-05-03 14:21:13 +0200 (Sat, 03 May 2008) | 8 lines In test_io, StatefulIncrementalDecoderTest was not part of the test suite. And of course, the test failed: a bytearray was used without reason in io.TextIOWrapper.tell(). The difference is that iterating over bytes (i.e. str in python2.6) returns 1-char bytes, whereas bytearrays yield integers. This code should still work with python3.0 ................ r62663 | benjamin.peterson | 2008-05-03 17:56:42 +0200 (Sat, 03 May 2008) | 2 lines The compiling struct is now passed around to all AST helpers (see issue 2720) ................ r62680 | benjamin.peterson | 2008-05-03 23:35:18 +0200 (Sat, 03 May 2008) | 2 lines Moved testing of builtin types out of test_builtin and into type specific modules ................ r62686 | mark.dickinson | 2008-05-04 04:25:46 +0200 (Sun, 04 May 2008) | 4 lines Make sure that Context traps and flags dictionaries have values 0 and 1 (as documented) rather than True and False. ................ r62687 | benjamin.peterson | 2008-05-04 05:05:49 +0200 (Sun, 04 May 2008) | 2 lines Fix typo in whatsnew ................ r62696 | georg.brandl | 2008-05-04 11:15:04 +0200 (Sun, 04 May 2008) | 2 lines #2752: wrong meaning of '' for socket host. ................ r62699 | christian.heimes | 2008-05-04 13:50:53 +0200 (Sun, 04 May 2008) | 1 line Added note that Python requires at least Win2k SP4 ................ r62700 | gerhard.haering | 2008-05-04 14:59:57 +0200 (Sun, 04 May 2008) | 3 lines SQLite requires 64-bit integers in order to build. So the whole HAVE_LONG_LONG #ifdefing was useless. ................ r62701 | gerhard.haering | 2008-05-04 15:15:12 +0200 (Sun, 04 May 2008) | 3 lines Applied sqliterow-richcmp.diff patch from Thomas Heller in Issue2152. The sqlite3.Row type is now correctly hashable. ................ r62702 | gerhard.haering | 2008-05-04 15:42:44 +0200 (Sun, 04 May 2008) | 5 lines Implemented feature request 2157: Converter names are cut off at '(' characters. This avoids the common case of something like 'NUMBER(10)' not being parsed as 'NUMBER', like expected. Also corrected the docs about converter names being case-sensitive. They aren't any longer. ................ r62703 | georg.brandl | 2008-05-04 17:45:05 +0200 (Sun, 04 May 2008) | 2 lines #2757: Remove spare newline. ................ r62711 | benjamin.peterson | 2008-05-04 21:10:02 +0200 (Sun, 04 May 2008) | 2 lines Fix typo in bugs.rst ................ Added: python/branches/py3k/Lib/ctypes/test/test_frombuffer.py - copied, changed from r62498, /python/trunk/Lib/ctypes/test/test_frombuffer.py python/branches/py3k/Lib/test/crashers/mutation_inside_cyclegc.py - copied unchanged from r62498, /python/trunk/Lib/test/crashers/mutation_inside_cyclegc.py python/branches/py3k/Lib/test/test_int.py - copied, changed from r62680, /python/trunk/Lib/test/test_int.py python/branches/py3k/Mac/Resources/app/Info.plist.in - copied unchanged from r62663, /python/trunk/Mac/Resources/app/Info.plist.in python/branches/py3k/Mac/Resources/framework/Info.plist.in - copied unchanged from r62663, /python/trunk/Mac/Resources/framework/Info.plist.in Removed: python/branches/py3k/Mac/Resources/app/Info.plist python/branches/py3k/Mac/Resources/app/Resources/English.lproj/InfoPlist.strings python/branches/py3k/Mac/Resources/framework/English.lproj/ python/branches/py3k/Mac/Resources/framework/Info.plist python/branches/py3k/Mac/Resources/framework/version.plist Modified: python/branches/py3k/ (props changed) python/branches/py3k/Doc/about.rst python/branches/py3k/Doc/bugs.rst python/branches/py3k/Doc/c-api/long.rst python/branches/py3k/Doc/c-api/number.rst python/branches/py3k/Doc/distutils/builtdist.rst python/branches/py3k/Doc/library/carbon.rst python/branches/py3k/Doc/library/ctypes.rst python/branches/py3k/Doc/library/getpass.rst python/branches/py3k/Doc/library/io.rst python/branches/py3k/Doc/library/optparse.rst python/branches/py3k/Doc/library/pkgutil.rst python/branches/py3k/Doc/library/pyclbr.rst python/branches/py3k/Doc/library/robotparser.rst python/branches/py3k/Doc/library/simplehttpserver.rst python/branches/py3k/Doc/library/socket.rst python/branches/py3k/Doc/library/sqlite3.rst python/branches/py3k/Doc/library/subprocess.rst python/branches/py3k/Doc/library/sys.rst python/branches/py3k/Doc/library/tempfile.rst python/branches/py3k/Doc/library/unittest.rst python/branches/py3k/Doc/library/xmlrpclib.rst python/branches/py3k/Doc/reference/expressions.rst python/branches/py3k/Doc/tools/sphinxext/indexcontent.html python/branches/py3k/Doc/using/cmdline.rst python/branches/py3k/Doc/whatsnew/2.6.rst python/branches/py3k/Lib/calendar.py python/branches/py3k/Lib/contextlib.py python/branches/py3k/Lib/decimal.py python/branches/py3k/Lib/distutils/command/bdist_wininst.py python/branches/py3k/Lib/distutils/command/wininst-6.0.exe python/branches/py3k/Lib/distutils/command/wininst-7.1.exe python/branches/py3k/Lib/distutils/command/wininst-9.0-amd64.exe python/branches/py3k/Lib/distutils/command/wininst-9.0.exe python/branches/py3k/Lib/distutils/msvc9compiler.py python/branches/py3k/Lib/getpass.py python/branches/py3k/Lib/idlelib/AutoComplete.py python/branches/py3k/Lib/idlelib/AutoCompleteWindow.py python/branches/py3k/Lib/idlelib/EditorWindow.py python/branches/py3k/Lib/idlelib/PyShell.py python/branches/py3k/Lib/idlelib/configHandler.py python/branches/py3k/Lib/lib2to3/ (props changed) python/branches/py3k/Lib/lib2to3/refactor.py python/branches/py3k/Lib/lib2to3/tests/test_fixers.py python/branches/py3k/Lib/plat-mac/Carbon/AppleEvents.py python/branches/py3k/Lib/plat-mac/macerrors.py python/branches/py3k/Lib/plat-mac/terminalcommand.py python/branches/py3k/Lib/pyclbr.py python/branches/py3k/Lib/robotparser.py python/branches/py3k/Lib/sqlite3/test/factory.py python/branches/py3k/Lib/sqlite3/test/types.py python/branches/py3k/Lib/tempfile.py python/branches/py3k/Lib/test/test_builtin.py python/branches/py3k/Lib/test/test_cmd_line_script.py python/branches/py3k/Lib/test/test_contextlib.py python/branches/py3k/Lib/test/test_decimal.py python/branches/py3k/Lib/test/test_float.py python/branches/py3k/Lib/test/test_frozen.py python/branches/py3k/Lib/test/test_import.py python/branches/py3k/Lib/test/test_io.py python/branches/py3k/Lib/test/test_list.py python/branches/py3k/Lib/test/test_long.py python/branches/py3k/Lib/test/test_sundry.py python/branches/py3k/Lib/test/test_support.py python/branches/py3k/Lib/test/test_textwrap.py python/branches/py3k/Lib/test/test_trace.py python/branches/py3k/Lib/test/test_traceback.py python/branches/py3k/Lib/test/test_tuple.py python/branches/py3k/Lib/test/test_urllib2.py python/branches/py3k/Lib/test/test_warnings.py python/branches/py3k/Lib/test/test_with.py python/branches/py3k/Lib/test/test_xmlrpc.py python/branches/py3k/Lib/textwrap.py python/branches/py3k/Lib/urllib2.py python/branches/py3k/Lib/weakref.py python/branches/py3k/Mac/BuildScript/build-installer.py python/branches/py3k/Mac/BuildScript/resources/ReadMe.txt python/branches/py3k/Mac/IDLE/Makefile.in python/branches/py3k/Mac/Makefile.in python/branches/py3k/Mac/PythonLauncher/Makefile.in python/branches/py3k/Mac/PythonLauncher/MyDocument.m python/branches/py3k/Mac/PythonLauncher/doscript.m python/branches/py3k/Makefile.pre.in python/branches/py3k/Misc/developers.txt python/branches/py3k/Modules/_ctypes/_ctypes.c python/branches/py3k/Modules/_sqlite/cursor.c python/branches/py3k/Modules/_sqlite/row.c python/branches/py3k/Modules/_sqlite/statement.c python/branches/py3k/Modules/_testcapimodule.c python/branches/py3k/Modules/mathmodule.c python/branches/py3k/PC/bdist_wininst/install.c python/branches/py3k/PC/bdist_wininst/wininst-7.1.vcproj python/branches/py3k/PC/bdist_wininst/wininst.dsp python/branches/py3k/Parser/tokenizer.c python/branches/py3k/Python/_warnings.c python/branches/py3k/Python/ast.c python/branches/py3k/Python/traceback.c python/branches/py3k/Tools/pybench/pybench.py python/branches/py3k/Tools/pybench/systimes.py python/branches/py3k/configure python/branches/py3k/configure.in python/branches/py3k/pyconfig.h.in Modified: python/branches/py3k/Doc/about.rst ============================================================================== --- python/branches/py3k/Doc/about.rst (original) +++ python/branches/py3k/Doc/about.rst Mon May 5 00:42:01 2008 @@ -18,8 +18,8 @@ * Fred L. Drake, Jr., the creator of the original Python documentation toolset and writer of much of the content; -* the `docutils `_ project for creating - reStructuredText and the docutils suite; +* the `Docutils `_ project for creating + reStructuredText and the Docutils suite; * Fredrik Lundh for his `Alternative Python Reference `_ project from which Sphinx got many good ideas. Modified: python/branches/py3k/Doc/bugs.rst ============================================================================== --- python/branches/py3k/Doc/bugs.rst (original) +++ python/branches/py3k/Doc/bugs.rst Mon May 5 00:42:01 2008 @@ -37,7 +37,7 @@ "Type" field, select the type of your problem; also select the "Component" and "Versions" to which the bug relates. -In the "Change Note" field, describe the problem in detail, including what you +In the "Comment" field, describe the problem in detail, including what you expected to happen and what did happen. Be sure to include whether any extension modules were involved, and what hardware and software platform you were using (including version information as appropriate). Modified: python/branches/py3k/Doc/c-api/long.rst ============================================================================== --- python/branches/py3k/Doc/c-api/long.rst (original) +++ python/branches/py3k/Doc/c-api/long.rst Mon May 5 00:42:01 2008 @@ -52,14 +52,14 @@ .. cfunction:: PyObject* PyLong_FromSsize_t(Py_ssize_t v) - Return a new :ctype:`PyLongObject` object with a value of *v*, or *NULL* - on failure. + Return a new :ctype:`PyLongObject` object from a C :ctype:`Py_ssize_t`, or + *NULL* on failure. .. cfunction:: PyObject* PyLong_FromSize_t(size_t v) - Return a new :ctype:`PyLongObject` object with a value of *v*, or *NULL* - on failure. + Return a new :ctype:`PyLongObject` object from a C :ctype:`size_t`, or + *NULL* on failure. .. cfunction:: PyObject* PyLong_FromLongLong(PY_LONG_LONG v) @@ -127,6 +127,17 @@ If an exception is set because of type errors, also return -1. +.. cfunction:: Py_ssize_t PyLong_AsSsize_t(PyObject *pylong) + + .. index:: + single: PY_SSIZE_T_MAX + single: OverflowError (built-in exception) + + Return a C :ctype:`Py_ssize_t` representation of the contents of *pylong*. If + *pylong* is greater than :const:`PY_SSIZE_T_MAX`, an :exc:`OverflowError` is raised + and ``-1`` will be returned. + + .. cfunction:: unsigned long PyLong_AsUnsignedLong(PyObject *pylong) .. index:: Modified: python/branches/py3k/Doc/c-api/number.rst ============================================================================== --- python/branches/py3k/Doc/c-api/number.rst (original) +++ python/branches/py3k/Doc/c-api/number.rst Mon May 5 00:42:01 2008 @@ -259,6 +259,17 @@ TypeError exception raised on failure. +.. cfunction:: PyObject* PyNumber_ToBase(PyObject *n, int base) + + Returns the the integer *n* converted to *base* as a string with a base + marker of ``'0b'``, ``'0o'``, or ``'0x'`` if appended applicable. When + *base* is not 2, 8, 10, or 16, the format is ``'x#num'`` where x is the + base. If *n* is not an int object, it is converted with + :cfunc:`PyNumber_Index` first. + + .. versionadded:: 2.6 + + .. cfunction:: Py_ssize_t PyNumber_AsSsize_t(PyObject *o, PyObject *exc) Returns *o* converted to a Py_ssize_t value if *o* can be interpreted as an Modified: python/branches/py3k/Doc/distutils/builtdist.rst ============================================================================== --- python/branches/py3k/Doc/distutils/builtdist.rst (original) +++ python/branches/py3k/Doc/distutils/builtdist.rst Mon May 5 00:42:01 2008 @@ -426,6 +426,13 @@ also the configuration. For details refer to Microsoft's documentation of the :cfunc:`SHGetSpecialFolderPath` function. +Vista User Access Control (UAC) +=============================== + +Starting with Python 2.6, bdist_wininst supports a :option:`--user-access-control` +option. The default is 'none' (meaning no UAC handling is done), and other +valid values are 'auto' (meaning prompt for UAC elevation if Python was +installed for all users) and 'force' (meaning always prompt for elevation) .. function:: create_shortcut(target, description, filename[, arguments[, workdir[, iconpath[, iconindex]]]]) @@ -437,5 +444,3 @@ and *iconindex* is the index of the icon in the file *iconpath*. Again, for details consult the Microsoft documentation for the :class:`IShellLink` interface. - - Modified: python/branches/py3k/Doc/library/carbon.rst ============================================================================== --- python/branches/py3k/Doc/library/carbon.rst (original) +++ python/branches/py3k/Doc/library/carbon.rst Mon May 5 00:42:01 2008 @@ -70,7 +70,7 @@ .. module:: Carbon.CG :platform: Mac - :synopsis: Interface to the Component Manager. + :synopsis: Interface to Core Graphics. Modified: python/branches/py3k/Doc/library/ctypes.rst ============================================================================== --- python/branches/py3k/Doc/library/ctypes.rst (original) +++ python/branches/py3k/Doc/library/ctypes.rst Mon May 5 00:42:01 2008 @@ -1948,6 +1948,28 @@ exact, they are methods of the :term:`metaclass`): + .. method:: _CData.from_buffer(source[, offset]) + + This method returns a ctypes instance that shares the buffer of + the ``source`` object. The ``source`` object must support the + writeable buffer interface. The optional ``offset`` parameter + specifies an offset into the source buffer in bytes; the default + is zero. If the source buffer is not large enough a ValueError + is raised. + + .. versionadded:: 2.6 + + .. method:: _CData.from_buffer_copy(source[, offset]) + + This method creates a ctypes instance, the buffer is copied from + the source object buffer which must be readable. The optional + ``offset`` parameter specifies an offset into the source buffer + in bytes; the default is zero. If the source buffer is not + large enough a ValueError is raised. + + .. versionadded:: 2.6 + + .. method:: from_address(address) This method returns a ctypes type instance using the memory specified by Modified: python/branches/py3k/Doc/library/getpass.rst ============================================================================== --- python/branches/py3k/Doc/library/getpass.rst (original) +++ python/branches/py3k/Doc/library/getpass.rst Mon May 5 00:42:01 2008 @@ -14,11 +14,27 @@ Prompt the user for a password without echoing. The user is prompted using the string *prompt*, which defaults to ``'Password: '``. On Unix, the prompt is - written to the file-like object *stream*, which defaults to ``sys.stdout`` (this - argument is ignored on Windows). + written to the file-like object *stream*. *stream* defaults to the + controlling terminal (/dev/tty) or if that is unavailable to ``sys.stderr`` + (this argument is ignored on Windows). + + If echo free input is unavailable getpass() falls back to printing + a warning message to *stream* and reading from ``sys.stdin`` and + issuing a :exc:`GetPassWarning`. Availability: Macintosh, Unix, Windows. + .. versionchanged:: 2.6 + On Unix it defaults to using /dev/tty before falling back + to ``sys.stdin`` and ``sys.stderr``. + .. note:: + If you call getpass from within IDLE, the input may be done in the + terminal you launched IDLE from rather than the idle window itself. + +.. exception:: GetPassWarning + + A :exc:`UserWarning` subclass issued when password input may be echoed. + .. function:: getuser() Modified: python/branches/py3k/Doc/library/io.rst ============================================================================== --- python/branches/py3k/Doc/library/io.rst (original) +++ python/branches/py3k/Doc/library/io.rst Mon May 5 00:42:01 2008 @@ -222,7 +222,7 @@ .. method:: fileno() - Return the underlying file descriptor (an integer) of the stream, if it + Return the underlying file descriptor (an integer) of the stream if it exists. An :exc:`IOError` is raised if the IO object does not use a file descriptor. @@ -233,18 +233,18 @@ .. method:: isatty() - Returns ``True`` if the stream is interactive (i.e., connected to + Return ``True`` if the stream is interactive (i.e., connected to a terminal/tty device). .. method:: readable() - Returns ``True`` if the stream can be read from. If False, - :meth:`read` will raise :exc:`IOError`. + Return ``True`` if the stream can be read from. If False, :meth:`read` + will raise :exc:`IOError`. .. method:: readline([limit]) - Reads and returns one line from the stream. If *limit* is - specified, at most *limit* bytes will be read. + Read and return one line from the stream. If *limit* is specified, at + most *limit* bytes will be read. The line terminator is always ``b'\n'`` for binary files; for text files, the *newlines* argument to :func:`open` can be used to select the line @@ -252,9 +252,9 @@ .. method:: readlines([hint]) - Returns a list of lines from the stream. *hint* can be specified to - control the number of lines read: no more lines will be read if the total - size (in bytes/characters) of all lines so far exceeds *hint*. + Read and return a list of lines from the stream. *hint* can be specified + to control the number of lines read: no more lines will be read if the + total size (in bytes/characters) of all lines so far exceeds *hint*. .. method:: seek(offset[, whence]) @@ -266,33 +266,32 @@ * ``1`` -- current stream position; *offset* may be negative * ``2`` -- end of the stream; *offset* is usually negative - Returns the new absolute position. + Return the new absolute position. .. method:: seekable() - Returns ``True`` if the stream supports random access. If - ``False``, :meth:`seek`, :meth:`tell` and :meth:`truncate` will - raise :exc:`IOError`. + Return ``True`` if the stream supports random access. If ``False``, + :meth:`seek`, :meth:`tell` and :meth:`truncate` will raise :exc:`IOError`. .. method:: tell() - Returns the current stream position. + Return the current stream position. .. method:: truncate([size]) - Truncates the file to at most *size* bytes. *size* defaults to the current + Truncate the file to at most *size* bytes. *size* defaults to the current file position, as returned by :meth:`tell`. .. method:: writable() - Returns ``True`` if the stream supports writing. If ``False``, + Return ``True`` if the stream supports writing. If ``False``, :meth:`write` and :meth:`truncate` will raise :exc:`IOError`. .. method:: writelines(lines) - Writes a list of lines to the stream. Line separators are not - added, so it is usual for each of the lines provided to have a - line separator at the end. + Write a list of lines to the stream. Line separators are not added, so it + is usual for each of the lines provided to have a line separator at the + end. .. class:: RawIOBase @@ -305,27 +304,26 @@ .. method:: read([n]) - Reads and returns all the bytes from the stream until EOF, or if *n* is - specified, up to *n* bytes. An empty bytes object is returned on EOF; - ``None`` is returned if the object is set not to block and has no data to - read. + Read and return all the bytes from the stream until EOF, or if *n* is + specified, up to *n* bytes. Only one system call is ever made. An empty + bytes object is returned on EOF; ``None`` is returned if the object is set + not to block and has no data to read. .. method:: readall() - Reads and returns all the bytes from the stream until EOF, using - multiple calls to the stream if necessary. + Read and return all the bytes from the stream until EOF, using multiple + calls to the stream if necessary. .. method:: readinto(b) - Reads up to len(b) bytes into bytearray *b* and returns the number - of bytes read. + Read up to len(b) bytes into bytearray *b* and return the number of bytes + read. .. method:: write(b) - Writes the given bytes or bytearray object, *b*, to the underlying - raw stream and returns the number of bytes written (never less - than ``len(b)``, since if the write fails an :exc:`IOError` will - be raised). + Write the given bytes or bytearray object, *b*, to the underlying raw + stream and return the number of bytes written (This is never less than + ``len(b)``, since if the write fails, an :exc:`IOError` will be raised). Raw File I/O @@ -352,22 +350,21 @@ .. attribute:: name - The file name. + The file name. This is the file descriptor of the file when no name is + given in the constructor. .. method:: read([n]) - Reads and returns at most *n* bytes. Only one system call is made, so - it is possible that less data than was requested is returned. Call - :func:`len` on the returned bytes object to see how many bytes - were actually returned (In non-blocking mode, ``None`` is returned - when no data is available.) + Read and return at most *n* bytes. Only one system call is made, so it is + possible that less data than was requested is returned. Use :func:`len` + on the returned bytes object to see how many bytes were actually returned. + (In non-blocking mode, ``None`` is returned when no data is available.) .. method:: readall() - Reads and returns the entire file's contents in a single bytes - object. As much as immediately available is returned in - non-blocking mode. If the EOF has been reached, ``b''`` is - returned. + Read and return the entire file's contents in a single bytes object. As + much as immediately available is returned in non-blocking mode. If the + EOF has been reached, ``b''`` is returned. .. method:: write(b) @@ -405,7 +402,7 @@ .. method:: read([n]) - Reads and returns up to *n* bytes. If the argument is omitted, ``None``, or + Read and return up to *n* bytes. If the argument is omitted, ``None``, or negative, data is read and returned until EOF is reached. An empty bytes object is returned if the stream is already at EOF. @@ -420,7 +417,7 @@ .. method:: readinto(b) - Reads up to len(b) bytes into bytearray *b* and returns the number of bytes + Read up to len(b) bytes into bytearray *b* and return the number of bytes read. Like :meth:`read`, multiple reads may be issued to the underlying raw @@ -431,10 +428,9 @@ .. method:: write(b) - Writes the given bytes or bytearray object, *b*, to the underlying - raw stream and returns the number of bytes written (never less than - ``len(b)``, since if the write fails an :exc:`IOError` will - be raised). + Write the given bytes or bytearray object, *b*, to the underlying raw + stream and return the number of bytes written (never less than ``len(b)``, + since if the write fails an :exc:`IOError` will be raised). A :exc:`BlockingIOError` is raised if the buffer is full, and the underlying raw stream cannot accept more data at the moment. @@ -452,8 +448,7 @@ .. method:: getvalue() - Returns a bytes object containing the entire contents of the - buffer. + Return ``bytes`` containing the entire contents of the buffer. .. method:: read1() @@ -461,8 +456,8 @@ .. method:: truncate([size]) - Truncates the buffer to at most *size* bytes. *size* defaults to the current - stream position, as returned by :meth:`tell`. + Truncate the buffer to at most *size* bytes. *size* defaults to the + current stream position, as returned by :meth:`tell`. .. class:: BufferedReader(raw[, buffer_size]) @@ -479,20 +474,20 @@ .. method:: peek([n]) - Returns 1 (or *n* if specified) bytes from a buffer without - advancing the position. Only a single read on the raw stream is done to - satisfy the call. The number of bytes returned may be less than - requested since at most all the buffer's bytes from the current - position to the end are returned. + Return 1 (or *n* if specified) bytes from a buffer without advancing the + position. Only a single read on the raw stream is done to satisfy the + call. The number of bytes returned may be less than requested since at + most all the buffer's bytes from the current position to the end are + returned. .. method:: read([n]) - Reads and returns *n* bytes, or if *n* is not given or negative, until EOF + Read and return *n* bytes, or if *n* is not given or negative, until EOF or if the read call would block in non-blocking mode. .. method:: read1(n) - Reads and returns up to *n* bytes with only one call on the raw stream. If + Read and return up to *n* bytes with only one call on the raw stream. If at least one byte is buffered, only buffered bytes are returned. Otherwise, one raw stream read call is made. @@ -517,9 +512,9 @@ .. method:: write(b) - Writes the bytes or bytearray object, *b*, onto the raw stream and - returns the number of bytes written. A :exc:`BlockingIOError` is - raised when the raw stream blocks. + Write the bytes or bytearray object, *b*, onto the raw stream and return + the number of bytes written. A :exc:`BlockingIOError` is raised when the + raw stream blocks. .. class:: BufferedRWPair(reader, writer[, buffer_size[, max_buffer_size]]) @@ -576,18 +571,18 @@ .. method:: read(n) - Reads and returns at most *n* characters from the stream as a - single :class:`str`. If *n* is negative or ``None``, reads to EOF. + Read and return at most *n* characters from the stream as a single + :class:`str`. If *n* is negative or ``None``, reads to EOF. .. method:: readline() - Reads until newline or EOF and returns a single :class:`str`. If - the stream is already at EOF, an empty string is returned. + Read until newline or EOF and return a single ``str``. If the stream is + already at EOF, an empty string is returned. .. method:: write(s) - Writes the string *s* to the stream and returns the number of - characters written. + Write the string *s* to the stream and return the number of characters + written. .. class:: TextIOWrapper(buffer[, encoding[, errors[, newline[, line_buffering]]]]) @@ -646,7 +641,7 @@ .. method:: getvalue() - Returns a :class:`str` containing the entire contents of the buffer. + Return a ``str`` containing the entire contents of the buffer. .. class:: IncrementalNewlineDecoder Modified: python/branches/py3k/Doc/library/optparse.rst ============================================================================== --- python/branches/py3k/Doc/library/optparse.rst (original) +++ python/branches/py3k/Doc/library/optparse.rst Mon May 5 00:42:01 2008 @@ -1633,7 +1633,7 @@ [...] parser.add_option("-c", "--callback", - action="callback", callback=varargs) + action="callback", callback=vararg_callback) The main weakness with this particular implementation is that negative numbers in the arguments following ``"-c"`` will be interpreted as further options Modified: python/branches/py3k/Doc/library/pkgutil.rst ============================================================================== --- python/branches/py3k/Doc/library/pkgutil.rst (original) +++ python/branches/py3k/Doc/library/pkgutil.rst Mon May 5 00:42:01 2008 @@ -42,13 +42,13 @@ Get a resource from a package. - This is a wrapper round the PEP 302 loader :func:`get_data` API. The package + This is a wrapper for the PEP 302 loader :func:`get_data` API. The package argument should be the name of a package, in standard module format (foo.bar). The resource argument should be in the form of a relative filename, using ``/`` as the path separator. The parent directory name ``..`` is not allowed, and nor is a rooted name (starting with a ``/``). - The function returns a binary string, which is the contents of the + The function returns a binary string that is the contents of the specified resource. For packages located in the filesystem, which have already been imported, Modified: python/branches/py3k/Doc/library/pyclbr.rst ============================================================================== --- python/branches/py3k/Doc/library/pyclbr.rst (original) +++ python/branches/py3k/Doc/library/pyclbr.rst Mon May 5 00:42:01 2008 @@ -7,75 +7,75 @@ .. sectionauthor:: Fred L. Drake, Jr. -The :mod:`pyclbr` can be used to determine some limited information about the -classes, methods and top-level functions defined in a module. The information -provided is sufficient to implement a traditional three-pane class browser. The -information is extracted from the source code rather than by importing the -module, so this module is safe to use with untrusted source code. This -restriction makes it impossible to use this module with modules not implemented -in Python, including many standard and optional extension modules. - - -.. function:: readmodule(module[, path]) - - Read a module and return a dictionary mapping class names to class descriptor - objects. The parameter *module* should be the name of a module as a string; - it may be the name of a module within a package. The *path* parameter should - be a sequence, and is used to augment the value of ``sys.path``, which is - used to locate module source code. - - .. The 'inpackage' parameter appears to be for internal use only.... - - -.. function:: readmodule_ex(module[, path]) - - Like :func:`readmodule`, but the returned dictionary, in addition to mapping - class names to class descriptor objects, also maps top-level function names to - function descriptor objects. Moreover, if the module being read is a package, - the key ``'__path__'`` in the returned dictionary has as its value a list which - contains the package search path. - - .. The 'inpackage' parameter appears to be for internal use only.... +The :mod:`pyclbr` module can be used to determine some limited information +about the classes, methods and top-level functions defined in a module. The +information provided is sufficient to implement a traditional three-pane +class browser. The information is extracted from the source code rather +than by importing the module, so this module is safe to use with untrusted +code. This restriction makes it impossible to use this module with modules +not implemented in Python, including all standard and optional extension +modules. + + +.. function:: readmodule(module[, path=None]) + + Read a module and return a dictionary mapping class names to class + descriptor objects. The parameter *module* should be the name of a + module as a string; it may be the name of a module within a package. The + *path* parameter should be a sequence, and is used to augment the value + of ``sys.path``, which is used to locate module source code. + + +.. function:: readmodule_ex(module[, path=None]) + + Like :func:`readmodule`, but the returned dictionary, in addition to + mapping class names to class descriptor objects, also maps top-level + function names to function descriptor objects. Moreover, if the module + being read is a package, the key ``'__path__'`` in the returned + dictionary has as its value a list which contains the package search + path. .. _pyclbr-class-objects: -Class Descriptor Objects ------------------------- +Class Objects +------------- -The class descriptor objects used as values in the dictionary returned by -:func:`readmodule` and :func:`readmodule_ex` provide the following data members: +The :class:`Class` objects used as values in the dictionary returned by +:func:`readmodule` and :func:`readmodule_ex` provide the following data +members: -.. attribute:: class_descriptor.module +.. attribute:: Class.module The name of the module defining the class described by the class descriptor. -.. attribute:: class_descriptor.name +.. attribute:: Class.name The name of the class. -.. attribute:: class_descriptor.super +.. attribute:: Class.super - A list of class descriptors which describe the immediate base classes of the - class being described. Classes which are named as superclasses but which are - not discoverable by :func:`readmodule` are listed as a string with the class - name instead of class descriptors. + A list of :class:`Class` objects which describe the immediate base + classes of the class being described. Classes which are named as + superclasses but which are not discoverable by :func:`readmodule` are + listed as a string with the class name instead of as :class:`Class` + objects. -.. attribute:: class_descriptor.methods +.. attribute:: Class.methods A dictionary mapping method names to line numbers. -.. attribute:: class_descriptor.file +.. attribute:: Class.file Name of the file containing the ``class`` statement defining the class. -.. attribute:: class_descriptor.lineno +.. attribute:: Class.lineno The line number of the ``class`` statement within the file named by :attr:`file`. @@ -83,30 +83,31 @@ .. _pyclbr-function-objects: -Function Descriptor Objects ---------------------------- +Function Objects +---------------- -The function descriptor objects used as values in the dictionary returned by +The :class:`Function` objects used as values in the dictionary returned by :func:`readmodule_ex` provide the following data members: -.. attribute:: function_descriptor.module +.. attribute:: Function.module The name of the module defining the function described by the function descriptor. -.. attribute:: function_descriptor.name +.. attribute:: Function.name The name of the function. -.. attribute:: function_descriptor.file +.. attribute:: Function.file Name of the file containing the ``def`` statement defining the function. -.. attribute:: function_descriptor.lineno +.. attribute:: Function.lineno - The line number of the ``def`` statement within the file named by :attr:`file`. + The line number of the ``def`` statement within the file named by + :attr:`file`. Modified: python/branches/py3k/Doc/library/robotparser.rst ============================================================================== --- python/branches/py3k/Doc/library/robotparser.rst (original) +++ python/branches/py3k/Doc/library/robotparser.rst Mon May 5 00:42:01 2008 @@ -3,7 +3,8 @@ ============================================= .. module:: robotparser - :synopsis: Loads a robots.txt file and answers questions about fetchability of other URLs. + :synopsis: Loads a robots.txt file and answers questions about + fetchability of other URLs. .. sectionauthor:: Skip Montanaro @@ -21,8 +22,8 @@ .. class:: RobotFileParser() - This class provides a set of methods to read, parse and answer questions about a - single :file:`robots.txt` file. + This class provides a set of methods to read, parse and answer questions + about a single :file:`robots.txt` file. .. method:: set_url(url) @@ -42,20 +43,22 @@ .. method:: can_fetch(useragent, url) - Returns ``True`` if the *useragent* is allowed to fetch the *url* according to - the rules contained in the parsed :file:`robots.txt` file. + Returns ``True`` if the *useragent* is allowed to fetch the *url* + according to the rules contained in the parsed :file:`robots.txt` + file. .. method:: mtime() - Returns the time the ``robots.txt`` file was last fetched. This is useful for - long-running web spiders that need to check for new ``robots.txt`` files - periodically. + Returns the time the ``robots.txt`` file was last fetched. This is + useful for long-running web spiders that need to check for new + ``robots.txt`` files periodically. .. method:: modified() - Sets the time the ``robots.txt`` file was last fetched to the current time. + Sets the time the ``robots.txt`` file was last fetched to the current + time. The following example demonstrates basic use of the RobotFileParser class. :: Modified: python/branches/py3k/Doc/library/simplehttpserver.rst ============================================================================== --- python/branches/py3k/Doc/library/simplehttpserver.rst (original) +++ python/branches/py3k/Doc/library/simplehttpserver.rst Mon May 5 00:42:01 2008 @@ -7,39 +7,40 @@ .. sectionauthor:: Moshe Zadka -The :mod:`SimpleHTTPServer` module defines a request-handler class, -interface-compatible with :class:`BaseHTTPServer.BaseHTTPRequestHandler`, that -serves files only from a base directory. +The :mod:`SimpleHTTPServer` module defines a single class, +:class:`SimpleHTTPRequestHandler`, which is interface-compatible with +:class:`BaseHTTPServer.BaseHTTPRequestHandler`. The :mod:`SimpleHTTPServer` module defines the following class: .. class:: SimpleHTTPRequestHandler(request, client_address, server) - This class is used to serve files from the current directory and below, directly + This class serves files from the current directory and below, directly mapping the directory structure to HTTP requests. A lot of the work, such as parsing the request, is done by the base class :class:`BaseHTTPServer.BaseHTTPRequestHandler`. This class implements the :func:`do_GET` and :func:`do_HEAD` functions. - The :class:`SimpleHTTPRequestHandler` defines the following member variables: + The following are defined as class-level attributes of + :class:`SimpleHTTPRequestHandler`: .. attribute:: server_version - This will be ``"SimpleHTTP/" + __version__``, where ``__version__`` is - defined in the module. + This will be ``"SimpleHTTP/" + __version__``, where ``__version__`` is + defined at the module level. .. attribute:: extensions_map - A dictionary mapping suffixes into MIME types. The default is signified by - an empty string, and is considered to be ``application/octet-stream``. The - mapping is used case-insensitively, and so should contain only lower-cased - keys. + A dictionary mapping suffixes into MIME types. The default is + signified by an empty string, and is considered to be + ``application/octet-stream``. The mapping is used case-insensitively, + and so should contain only lower-cased keys. - The :class:`SimpleHTTPRequestHandler` defines the following methods: + The :class:`SimpleHTTPRequestHandler` class defines the following methods: .. method:: do_HEAD() Modified: python/branches/py3k/Doc/library/socket.rst ============================================================================== --- python/branches/py3k/Doc/library/socket.rst (original) +++ python/branches/py3k/Doc/library/socket.rst Mon May 5 00:42:01 2008 @@ -765,7 +765,7 @@ # Echo server program import socket - HOST = '' # Symbolic name meaning the local host + HOST = '' # Symbolic name meaning all available interfaces PORT = 50007 # Arbitrary non-privileged port s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind((HOST, PORT)) Modified: python/branches/py3k/Doc/library/sqlite3.rst ============================================================================== --- python/branches/py3k/Doc/library/sqlite3.rst (original) +++ python/branches/py3k/Doc/library/sqlite3.rst Mon May 5 00:42:01 2008 @@ -112,10 +112,11 @@ :func:`connect` function. Setting it makes the :mod:`sqlite3` module parse the declared type for each - column it returns. It will parse out the first word of the declared type, i. e. - for "integer primary key", it will parse out "integer". Then for that column, it - will look into the converters dictionary and use the converter function - registered for that type there. Converter names are case-sensitive! + column it returns. It will parse out the first word of the declared type, + i. e. for "integer primary key", it will parse out "integer", or for + "number(10)" it will parse out "number". Then for that column, it will look + into the converters dictionary and use the converter function registered for + that type there. .. data:: PARSE_COLNAMES @@ -654,10 +655,6 @@ Converter functions **always** get called with a string, no matter under which data type you sent the value to SQLite. -.. note:: - - Converter names are looked up in a case-sensitive manner. - :: def convert_point(s): Modified: python/branches/py3k/Doc/library/subprocess.rst ============================================================================== --- python/branches/py3k/Doc/library/subprocess.rst (original) +++ python/branches/py3k/Doc/library/subprocess.rst Mon May 5 00:42:01 2008 @@ -216,7 +216,7 @@ .. method:: Popen.terminate() Stop the child. On Posix OSs the method sends SIGTERM to the - child. On Windows the Win32 API function TerminateProcess is called + child. On Windows the Win32 API function :cfunc:`TerminateProcess` is called to stop the child. Modified: python/branches/py3k/Doc/library/sys.rst ============================================================================== --- python/branches/py3k/Doc/library/sys.rst (original) +++ python/branches/py3k/Doc/library/sys.rst Mon May 5 00:42:01 2008 @@ -379,17 +379,17 @@ *platform* may be one of the following values: - +-----------------------------------------+-----------------------+ - | Constant | Platform | - +=========================================+=======================+ - | :const:`0 (VER_PLATFORM_WIN32s)` | Win32s on Windows 3.1 | - +-----------------------------------------+-----------------------+ - | :const:`1 (VER_PLATFORM_WIN32_WINDOWS)` | Windows 95/98/ME | - +-----------------------------------------+-----------------------+ - | :const:`2 (VER_PLATFORM_WIN32_NT)` | Windows NT/2000/XP | - +-----------------------------------------+-----------------------+ - | :const:`3 (VER_PLATFORM_WIN32_CE)` | Windows CE | - +-----------------------------------------+-----------------------+ + +-----------------------------------------+-------------------------+ + | Constant | Platform | + +=========================================+=========================+ + | :const:`0 (VER_PLATFORM_WIN32s)` | Win32s on Windows 3.1 | + +-----------------------------------------+-------------------------+ + | :const:`1 (VER_PLATFORM_WIN32_WINDOWS)` | Windows 95/98/ME | + +-----------------------------------------+-------------------------+ + | :const:`2 (VER_PLATFORM_WIN32_NT)` | Windows NT/2000/XP/x64 | + +-----------------------------------------+-------------------------+ + | :const:`3 (VER_PLATFORM_WIN32_CE)` | Windows CE | + +-----------------------------------------+-------------------------+ This function wraps the Win32 :cfunc:`GetVersionEx` function; see the Microsoft documentation for more information about these fields. Modified: python/branches/py3k/Doc/library/tempfile.rst ============================================================================== --- python/branches/py3k/Doc/library/tempfile.rst (original) +++ python/branches/py3k/Doc/library/tempfile.rst Mon May 5 00:42:01 2008 @@ -23,147 +23,155 @@ no longer contain the process ID; instead a string of six random characters is used. -Also, all the user-callable functions now take additional arguments which allow -direct control over the location and name of temporary files. It is no longer -necessary to use the global *tempdir* and *template* variables. To maintain -backward compatibility, the argument order is somewhat odd; it is recommended to -use keyword arguments for clarity. +Also, all the user-callable functions now take additional arguments which +allow direct control over the location and name of temporary files. It is +no longer necessary to use the global *tempdir* and *template* variables. +To maintain backward compatibility, the argument order is somewhat odd; it +is recommended to use keyword arguments for clarity. The module defines the following user-callable functions: -.. function:: TemporaryFile([mode='w+b'[, bufsize=-1[, suffix[, prefix[, dir]]]]]) +.. function:: TemporaryFile([mode='w+b'[, bufsize=-1[, suffix=''[, prefix='tmp'[, dir=None]]]]]) - Return a file-like object that can be used as a temporary storage - area. The file is created using :func:`mkstemp`. It will be destroyed as soon + Return a file-like object that can be used as a temporary storage area. + The file is created using :func:`mkstemp`. It will be destroyed as soon as it is closed (including an implicit close when the object is garbage - collected). Under Unix, the directory entry for the file is removed immediately - after the file is created. Other platforms do not support this; your code - should not rely on a temporary file created using this function having or not - having a visible name in the file system. - - The *mode* parameter defaults to ``'w+b'`` so that the file created can be read - and written without being closed. Binary mode is used so that it behaves - consistently on all platforms without regard for the data that is stored. - *bufsize* defaults to ``-1``, meaning that the operating system default is used. + collected). Under Unix, the directory entry for the file is removed + immediately after the file is created. Other platforms do not support + this; your code should not rely on a temporary file created using this + function having or not having a visible name in the file system. + + The *mode* parameter defaults to ``'w+b'`` so that the file created can + be read and written without being closed. Binary mode is used so that it + behaves consistently on all platforms without regard for the data that is + stored. *bufsize* defaults to ``-1``, meaning that the operating system + default is used. The *dir*, *prefix* and *suffix* parameters are passed to :func:`mkstemp`. The returned object is a true file object on POSIX platforms. On other platforms, it is a file-like object whose :attr:`file` attribute is the - underlying true file object. This file-like object can be used in a :keyword:`with` - statement, just like a normal file. - - -.. function:: NamedTemporaryFile([mode='w+b'[, bufsize=-1[, suffix[, prefix[, dir[, delete]]]]]]) - - This function operates exactly as :func:`TemporaryFile` does, except that the - file is guaranteed to have a visible name in the file system (on Unix, the - directory entry is not unlinked). That name can be retrieved from the - :attr:`name` member of the file object. Whether the name can be used to open - the file a second time, while the named temporary file is still open, varies - across platforms (it can be so used on Unix; it cannot on Windows NT or later). - If *delete* is true (the default), the file is deleted as soon as it is closed. - The returned object is always a file-like object whose :attr:`file` attribute - is the underlying true file object. This file-like object can be used in a :keyword:`with` - statement, just like a normal file. + underlying true file object. This file-like object can be used in a + :keyword:`with` statement, just like a normal file. -.. function:: SpooledTemporaryFile([max_size=0, [mode='w+b'[, bufsize=-1[, suffix[, prefix[, dir]]]]]]) +.. function:: NamedTemporaryFile([mode='w+b'[, bufsize=-1[, suffix=''[, prefix='tmp'[, dir=None[, delete=True]]]]]]) - This function operates exactly as :func:`TemporaryFile` does, except that data - is spooled in memory until the file size exceeds *max_size*, or until the file's - :func:`fileno` method is called, at which point the contents are written to disk - and operation proceeds as with :func:`TemporaryFile`. + This function operates exactly as :func:`TemporaryFile` does, except that + the file is guaranteed to have a visible name in the file system (on + Unix, the directory entry is not unlinked). That name can be retrieved + from the :attr:`name` member of the file object. Whether the name can be + used to open the file a second time, while the named temporary file is + still open, varies across platforms (it can be so used on Unix; it cannot + on Windows NT or later). If *delete* is true (the default), the file is + deleted as soon as it is closed. + The returned object is always a file-like object whose :attr:`file` + attribute is the underlying true file object. This file-like object can + be used in a :keyword:`with` statement, just like a normal file. + + +.. function:: SpooledTemporaryFile([max_size=0, [mode='w+b'[, bufsize=-1[, suffix=''[, prefix='tmp'[, dir=None]]]]]]) + + This function operates exactly as :func:`TemporaryFile` does, except that + data is spooled in memory until the file size exceeds *max_size*, or + until the file's :func:`fileno` method is called, at which point the + contents are written to disk and operation proceeds as with + :func:`TemporaryFile`. - The resulting file has one additional method, :func:`rollover`, which causes the - file to roll over to an on-disk file regardless of its size. + The resulting file has one additional method, :func:`rollover`, which + causes the file to roll over to an on-disk file regardless of its size. The returned object is a file-like object whose :attr:`_file` attribute is either a :class:`StringIO` object or a true file object, depending on - whether :func:`rollover` has been called. This file-like object can be used in a - :keyword:`with` statement, just like a normal file. + whether :func:`rollover` has been called. This file-like object can be + used in a :keyword:`with` statement, just like a normal file. -.. function:: mkstemp([suffix[, prefix[, dir[, text]]]]) +.. function:: mkstemp([suffix=''[, prefix='tmp'[, dir=None[, text=False]]]]) - Creates a temporary file in the most secure manner possible. There are no - race conditions in the file's creation, assuming that the platform properly - implements the :const:`os.O_EXCL` flag for :func:`os.open`. The file is - readable and writable only by the creating user ID. If the platform uses - permission bits to indicate whether a file is executable, the file is - executable by no one. The file descriptor is not inherited by child - processes. + Creates a temporary file in the most secure manner possible. There are + no race conditions in the file's creation, assuming that the platform + properly implements the :const:`os.O_EXCL` flag for :func:`os.open`. The + file is readable and writable only by the creating user ID. If the + platform uses permission bits to indicate whether a file is executable, + the file is executable by no one. The file descriptor is not inherited + by child processes. - Unlike :func:`TemporaryFile`, the user of :func:`mkstemp` is responsible for - deleting the temporary file when done with it. + Unlike :func:`TemporaryFile`, the user of :func:`mkstemp` is responsible + for deleting the temporary file when done with it. - If *suffix* is specified, the file name will end with that suffix, otherwise - there will be no suffix. :func:`mkstemp` does not put a dot between the file - name and the suffix; if you need one, put it at the beginning of *suffix*. + If *suffix* is specified, the file name will end with that suffix, + otherwise there will be no suffix. :func:`mkstemp` does not put a dot + between the file name and the suffix; if you need one, put it at the + beginning of *suffix*. - If *prefix* is specified, the file name will begin with that prefix; otherwise, - a default prefix is used. + If *prefix* is specified, the file name will begin with that prefix; + otherwise, a default prefix is used. - If *dir* is specified, the file will be created in that directory; otherwise, - a default directory is used. The default directory is chosen from a - platform-dependent list, but the user of the application can control the - directory location by setting the *TMPDIR*, *TEMP* or *TMP* environment - variables. There is thus no guarantee that the generated filename will have - any nice properties, such as not requiring quoting when passed to external - commands via ``os.popen()``. + If *dir* is specified, the file will be created in that directory; + otherwise, a default directory is used. The default directory is chosen + from a platform-dependent list, but the user of the application can + control the directory location by setting the *TMPDIR*, *TEMP* or *TMP* + environment variables. There is thus no guarantee that the generated + filename will have any nice properties, such as not requiring quoting + when passed to external commands via ``os.popen()``. - If *text* is specified, it indicates whether to open the file in binary mode - (the default) or text mode. On some platforms, this makes no difference. + If *text* is specified, it indicates whether to open the file in binary + mode (the default) or text mode. On some platforms, this makes no + difference. - :func:`mkstemp` returns a tuple containing an OS-level handle to an open file - (as would be returned by :func:`os.open`) and the absolute pathname of that - file, in that order. + :func:`mkstemp` returns a tuple containing an OS-level handle to an open + file (as would be returned by :func:`os.open`) and the absolute pathname + of that file, in that order. -.. function:: mkdtemp([suffix[, prefix[, dir]]]) +.. function:: mkdtemp([suffix=''[, prefix='tmp'[, dir=None]]]) - Creates a temporary directory in the most secure manner possible. There are no - race conditions in the directory's creation. The directory is readable, - writable, and searchable only by the creating user ID. + Creates a temporary directory in the most secure manner possible. There + are no race conditions in the directory's creation. The directory is + readable, writable, and searchable only by the creating user ID. - The user of :func:`mkdtemp` is responsible for deleting the temporary directory - and its contents when done with it. + The user of :func:`mkdtemp` is responsible for deleting the temporary + directory and its contents when done with it. - The *prefix*, *suffix*, and *dir* arguments are the same as for :func:`mkstemp`. + The *prefix*, *suffix*, and *dir* arguments are the same as for + :func:`mkstemp`. :func:`mkdtemp` returns the absolute pathname of the new directory. -.. function:: mktemp([suffix[, prefix[, dir]]]) +.. function:: mktemp([suffix=''[, prefix='tmp'[, dir=None]]]) .. deprecated:: 2.3 Use :func:`mkstemp` instead. - Return an absolute pathname of a file that did not exist at the time the call is - made. The *prefix*, *suffix*, and *dir* arguments are the same as for - :func:`mkstemp`. + Return an absolute pathname of a file that did not exist at the time the + call is made. The *prefix*, *suffix*, and *dir* arguments are the same + as for :func:`mkstemp`. .. warning:: - Use of this function may introduce a security hole in your program. By the time - you get around to doing anything with the file name it returns, someone else may - have beaten you to the punch. - -The module uses two global variables that tell it how to construct a temporary -name. They are initialized at the first call to any of the functions above. -The caller may change them, but this is discouraged; use the appropriate -function arguments, instead. + Use of this function may introduce a security hole in your program. + By the time you get around to doing anything with the file name it + returns, someone else may have beaten you to the punch. + +The module uses two global variables that tell it how to construct a +temporary name. They are initialized at the first call to any of the +functions above. The caller may change them, but this is discouraged; use +the appropriate function arguments, instead. .. data:: tempdir - When set to a value other than ``None``, this variable defines the default value - for the *dir* argument to all the functions defined in this module. - - If ``tempdir`` is unset or ``None`` at any call to any of the above functions, - Python searches a standard list of directories and sets *tempdir* to the first - one which the calling user can create files in. The list is: + When set to a value other than ``None``, this variable defines the + default value for the *dir* argument to all the functions defined in this + module. + + If ``tempdir`` is unset or ``None`` at any call to any of the above + functions, Python searches a standard list of directories and sets + *tempdir* to the first one which the calling user can create files in. + The list is: #. The directory named by the :envvar:`TMPDIR` environment variable. Modified: python/branches/py3k/Doc/library/unittest.rst ============================================================================== --- python/branches/py3k/Doc/library/unittest.rst (original) +++ python/branches/py3k/Doc/library/unittest.rst Mon May 5 00:42:01 2008 @@ -260,7 +260,6 @@ Often, many small test cases will use the same fixture. In this case, we would end up subclassing :class:`SimpleWidgetTestCase` into many small one-method classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and - discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler mechanism:: Modified: python/branches/py3k/Doc/library/xmlrpclib.rst ============================================================================== --- python/branches/py3k/Doc/library/xmlrpclib.rst (original) +++ python/branches/py3k/Doc/library/xmlrpclib.rst Mon May 5 00:42:01 2008 @@ -111,6 +111,14 @@ `XML-RPC Introspection `_ Describes the XML-RPC protocol extension for introspection. + `XML-RPC Specification `_ + The official specification. + + `Unofficial XML-RPC Errata `_ + Fredrik Lundh's "unofficial errata, intended to clarify certain + details in the XML-RPC specification, as well as hint at + 'best practices' to use when designing your own XML-RPC + implementations." .. _serverproxy-objects: @@ -280,6 +288,11 @@ Write the XML-RPC base 64 encoding of this binary item to the out stream object. + The encoded data will have newlines every 76 characters as per + `RFC 2045 section 6.8 `_, + which was the de facto standard base64 specification when the + XML-RPC spec was written. + It also supports certain of Python's built-in operators through a :meth:`__cmp__` method. Modified: python/branches/py3k/Doc/reference/expressions.rst ============================================================================== --- python/branches/py3k/Doc/reference/expressions.rst (original) +++ python/branches/py3k/Doc/reference/expressions.rst Mon May 5 00:42:01 2008 @@ -651,6 +651,14 @@ raised. Otherwise, the list of filled slots is used as the argument list for the call. +.. note:: + + An implementation may provide builtin functions whose positional parameters do + not have names, even if they are 'named' for the purpose of documentation, and + which therefore cannot be supplied by keyword. In CPython, this is the case for + functions implemented in C that use :cfunc:`PyArg_ParseTuple` to parse their + arguments. + If there are more positional arguments than there are formal parameter slots, a :exc:`TypeError` exception is raised, unless a formal parameter using the syntax ``*identifier`` is present; in this case, that formal parameter receives a tuple Modified: python/branches/py3k/Doc/tools/sphinxext/indexcontent.html ============================================================================== --- python/branches/py3k/Doc/tools/sphinxext/indexcontent.html (original) +++ python/branches/py3k/Doc/tools/sphinxext/indexcontent.html Mon May 5 00:42:01 2008 @@ -3,28 +3,28 @@

Parts of the documentation:

-

What's new in Python {{ version }}?
+

What's new in Python {{ version }}?
changes since previous major release

-

Tutorial
+

Tutorial
start here

-

Using Python
+

Using Python
how to use Python on different platforms

-

Language Reference
+

Language Reference
describes syntax and language elements

-

Library Reference
+

Library Reference
keep this under your pillow

-

Python HOWTOs
+

Python HOWTOs
in-depth documents on specific topics

 -

Extending and Embedding
+

Extending and Embedding
tutorial for C/C++ programmers

-

Python/C API
+

Python/C API
reference for C/C++ programmers

-

Installing Python Modules
+

Installing Python Modules
information for installers & sys-admins

-

Distributing Python Modules
+

Distributing Python Modules
sharing modules with others

-

Documenting Python
+

Documenting Python
guide for documentation authors

@@ -32,16 +32,16 @@

Indices and tables:

-

Global Module Index
+

Global Module Index
quick access to all modules

-

General Index
+

General Index
all functions, classes, terms

-

Glossary
+

Glossary
the most important terms explained

 -

Search page
+

Search page
search this documentation

-

Complete Table of Contents
+

Complete Table of Contents
lists all sections and subsections

Modified: python/branches/py3k/Doc/using/cmdline.rst ============================================================================== --- python/branches/py3k/Doc/using/cmdline.rst (original) +++ python/branches/py3k/Doc/using/cmdline.rst Mon May 5 00:42:01 2008 @@ -28,20 +28,25 @@ python myscript.py +.. _using-on-interface-options: + Interface options ~~~~~~~~~~~~~~~~~ -The interpreter interface resembles that of the UNIX shell: +The interpreter interface resembles that of the UNIX shell, but provides some +additional methods of invocation: * When called with standard input connected to a tty device, it prompts for commands and executes them until an EOF (an end-of-file character, you can produce that with *Ctrl-D* on UNIX or *Ctrl-Z, Enter* on Windows) is read. * When called with a file name argument or with a file as standard input, it reads and executes a script from that file. +* When called with a directory name argument, it reads and executes an + appropriately named script from that directory. * When called with ``-c command``, it executes the Python statement(s) given as *command*. Here *command* may contain multiple statements separated by newlines. Leading whitespace is significant in Python statements! -* When called with ``-m module-name``, the given module is searched on the +* When called with ``-m module-name``, the given module is located on the Python module path and executed as a script. In non-interactive mode, the entire input is parsed before it is executed. @@ -58,25 +63,31 @@ normal module code. If this option is given, the first element of :data:`sys.argv` will be - ``"-c"``. + ``"-c"`` and the current directory will be added to the start of + :data:`sys.path` (allowing modules in that directory to be imported as top + level modules). .. cmdoption:: -m - Search :data:`sys.path` for the named module and run the corresponding module - file as if it were executed with ``python modulefile.py`` as a script. + Search :data:`sys.path` for the named module and execute its contents as + the :mod:`__main__` module. Since the argument is a *module* name, you must not give a file extension - (``.py``). However, the ``module-name`` does not have to be a valid Python - identifer (e.g. you can use a file name including a hyphen). + (``.py``). The ``module-name`` should be a valid Python module name, but + the implementation may not always enforce this (e.g. it may allow you to + use a name that includes a hyphen). .. note:: This option cannot be used with builtin modules and extension modules - written in C, since they do not have Python module files. + written in C, since they do not have Python module files. However, it + can still be used for precompiled modules, even if the original source + file is not available. If this option is given, the first element of :data:`sys.argv` will be the - full path to the module file. + full path to the module file. As with the :option:`-c` option, the current + directory will be added to the start of :data:`sys.path`. Many standard library modules contain code that is invoked on their execution as a script. An example is the :mod:`timeit` module:: @@ -91,28 +102,44 @@ :pep:`338` -- Executing modules as scripts -.. describe:: - """ % ( self.OutputString(attrs), ) - # end js_output() - - def OutputString(self, attrs=None): - # Build up our result - # - result = [] - RA = result.append - - # First, the key=value pair - RA("%s=%s" % (self.key, self.coded_value)) - - # Now add any defined attributes - if attrs is None: - attrs = self._reserved - items = sorted(self.items()) - for K,V in items: - if V == "": continue - if K not in attrs: continue - if K == "expires" and type(V) == type(1): - RA("%s=%s" % (self._reserved[K], _getdate(V))) - elif K == "max-age" and type(V) == type(1): - RA("%s=%d" % (self._reserved[K], V)) - elif K == "secure": - RA(str(self._reserved[K])) - else: - RA("%s=%s" % (self._reserved[K], V)) - - # Return the result - return _semispacejoin(result) - # end OutputString -# end Morsel class - - - -# -# Pattern for finding cookie -# -# This used to be strict parsing based on the RFC2109 and RFC2068 -# specifications. I have since discovered that MSIE 3.0x doesn't -# follow the character rules outlined in those specs. As a -# result, the parsing rules here are less strict. -# - -_LegalCharsPatt = r"[\w\d!#%&'~_`><@,:/\$\*\+\-\.\^\|\)\(\?\}\{\=]" -_CookiePattern = re.compile( - r"(?x)" # This is a Verbose pattern - r"(?P" # Start of group 'key' - ""+ _LegalCharsPatt +"+?" # Any word of at least one letter, nongreedy - r")" # End of group 'key' - r"\s*=\s*" # Equal Sign - r"(?P" # Start of group 'val' - r'"(?:[^\\"]|\\.)*"' # Any doublequoted string - r"|" # or - ""+ _LegalCharsPatt +"*" # Any word or empty string - r")" # End of group 'val' - r"\s*;?" # Probably ending in a semi-colon - ) - - -# At long last, here is the cookie class. -# Using this class is almost just like using a dictionary. -# See this module's docstring for example usage. -# -class BaseCookie(dict): - # A container class for a set of Morsels - # - - def value_decode(self, val): - """real_value, coded_value = value_decode(STRING) - Called prior to setting a cookie's value from the network - representation. The VALUE is the value read from HTTP - header. - Override this function to modify the behavior of cookies. - """ - return val, val - # end value_encode - - def value_encode(self, val): - """real_value, coded_value = value_encode(VALUE) - Called prior to setting a cookie's value from the dictionary - representation. The VALUE is the value being assigned. - Override this function to modify the behavior of cookies. - """ - strval = str(val) - return strval, strval - # end value_encode - - def __init__(self, input=None): - if input: self.load(input) - # end __init__ - - def __set(self, key, real_value, coded_value): - """Private method for setting a cookie's value""" - M = self.get(key, Morsel()) - M.set(key, real_value, coded_value) - dict.__setitem__(self, key, M) - # end __set - - def __setitem__(self, key, value): - """Dictionary style assignment.""" - rval, cval = self.value_encode(value) - self.__set(key, rval, cval) - # end __setitem__ - - def output(self, attrs=None, header="Set-Cookie:", sep="\015\012"): - """Return a string suitable for HTTP.""" - result = [] - items = sorted(self.items()) - for K,V in items: - result.append( V.output(attrs, header) ) - return sep.join(result) - # end output - - __str__ = output - - def __repr__(self): - L = [] - items = sorted(self.items()) - for K,V in items: - L.append( '%s=%s' % (K,repr(V.value) ) ) - return '<%s: %s>' % (self.__class__.__name__, _spacejoin(L)) - - def js_output(self, attrs=None): - """Return a string suitable for JavaScript.""" - result = [] - items = sorted(self.items()) - for K,V in items: - result.append( V.js_output(attrs) ) - return _nulljoin(result) - # end js_output - - def load(self, rawdata): - """Load cookies from a string (presumably HTTP_COOKIE) or - from a dictionary. Loading cookies from a dictionary 'd' - is equivalent to calling: - map(Cookie.__setitem__, d.keys(), d.values()) - """ - if type(rawdata) == type(""): - self.__ParseString(rawdata) - else: - self.update(rawdata) - return - # end load() - - def __ParseString(self, str, patt=_CookiePattern): - i = 0 # Our starting point - n = len(str) # Length of string - M = None # current morsel - - while 0 <= i < n: - # Start looking for a cookie - match = patt.search(str, i) - if not match: break # No more cookies - - K,V = match.group("key"), match.group("val") - i = match.end(0) - - # Parse the key, value in case it's metainfo - if K[0] == "$": - # We ignore attributes which pertain to the cookie - # mechanism as a whole. See RFC 2109. - # (Does anyone care?) - if M: - M[ K[1:] ] = V - elif K.lower() in Morsel._reserved: - if M: - M[ K ] = _unquote(V) - else: - rval, cval = self.value_decode(V) - self.__set(K, rval, cval) - M = self[K] - # end __ParseString -# end BaseCookie class - -class SimpleCookie(BaseCookie): - """SimpleCookie - SimpleCookie supports strings as cookie values. When setting - the value using the dictionary assignment notation, SimpleCookie - calls the builtin str() to convert the value to a string. Values - received from HTTP are kept as strings. - """ - def value_decode(self, val): - return _unquote( val ), val - def value_encode(self, val): - strval = str(val) - return strval, _quote( strval ) -# end SimpleCookie - -class SerialCookie(BaseCookie): - """SerialCookie - SerialCookie supports arbitrary objects as cookie values. All - values are serialized (using pickle) before being sent to the - client. All incoming values are assumed to be valid Pickle - representations. IF AN INCOMING VALUE IS NOT IN A VALID PICKLE - FORMAT, THEN AN EXCEPTION WILL BE RAISED. - - Note: Large cookie values add overhead because they must be - retransmitted on every HTTP transaction. - - Note: HTTP has a 2k limit on the size of a cookie. This class - does not check for this limit, so be careful!!! - """ - def __init__(self, input=None): - warnings.warn("SerialCookie class is insecure; do not use it", - DeprecationWarning) - BaseCookie.__init__(self, input) - # end __init__ - def value_decode(self, val): - # This could raise an exception! - return loads( _unquote(val).encode('latin-1') ), val - def value_encode(self, val): - return val, _quote( dumps(val, 0).decode('latin-1') ) -# end SerialCookie - -class SmartCookie(BaseCookie): - """SmartCookie - SmartCookie supports arbitrary objects as cookie values. If the - object is a string, then it is quoted. If the object is not a - string, however, then SmartCookie will use pickle to serialize - the object into a string representation. - - Note: Large cookie values add overhead because they must be - retransmitted on every HTTP transaction. - - Note: HTTP has a 2k limit on the size of a cookie. This class - does not check for this limit, so be careful!!! - """ - def __init__(self, input=None): - warnings.warn("Cookie/SmartCookie class is insecure; do not use it", - DeprecationWarning) - BaseCookie.__init__(self, input) - # end __init__ - def value_decode(self, val): - strval = _unquote(val) - try: - return loads(strval.encode('latin-1')), val - except: - return strval, val - def value_encode(self, val): - if isinstance(val, str): - return val, _quote(val) - else: - return val, _quote( dumps(val, 0).decode('latin-1') ) -# end SmartCookie - - -########################################################### -# Backwards Compatibility: Don't break any existing code! - -# We provide Cookie() as an alias for SmartCookie() -Cookie = SmartCookie - -# -########################################################### - -def _test(): - import doctest, Cookie - return doctest.testmod(Cookie) - -if __name__ == "__main__": - _test() - - -#Local Variables: -#tab-width: 4 -#end: Deleted: python/branches/py3k/Lib/SimpleHTTPServer.py ============================================================================== --- python/branches/py3k/Lib/SimpleHTTPServer.py Mon May 26 18:32:26 2008 +++ (empty file) @@ -1,216 +0,0 @@ -"""Simple HTTP Server. - -This module builds on BaseHTTPServer by implementing the standard GET -and HEAD requests in a fairly straightforward manner. - -""" - - -__version__ = "0.6" - -__all__ = ["SimpleHTTPRequestHandler"] - -import os -import sys -import posixpath -import BaseHTTPServer -import urllib -import cgi -import shutil -import mimetypes -from io import BytesIO - - -class SimpleHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): - - """Simple HTTP request handler with GET and HEAD commands. - - This serves files from the current directory and any of its - subdirectories. The MIME type for files is determined by - calling the .guess_type() method. - - The GET and HEAD requests are identical except that the HEAD - request omits the actual contents of the file. - - """ - - server_version = "SimpleHTTP/" + __version__ - - def do_GET(self): - """Serve a GET request.""" - f = self.send_head() - if f: - self.copyfile(f, self.wfile) - f.close() - - def do_HEAD(self): - """Serve a HEAD request.""" - f = self.send_head() - if f: - f.close() - - def send_head(self): - """Common code for GET and HEAD commands. - - This sends the response code and MIME headers. - - Return value is either a file object (which has to be copied - to the outputfile by the caller unless the command was HEAD, - and must be closed by the caller under all circumstances), or - None, in which case the caller has nothing further to do. - - """ - path = self.translate_path(self.path) - f = None - if os.path.isdir(path): - if not self.path.endswith('/'): - # redirect browser - doing basically what apache does - self.send_response(301) - self.send_header("Location", self.path + "/") - self.end_headers() - return None - for index in "index.html", "index.htm": - index = os.path.join(path, index) - if os.path.exists(index): - path = index - break - else: - return self.list_directory(path) - ctype = self.guess_type(path) - try: - f = open(path, 'rb') - except IOError: - self.send_error(404, "File not found") - return None - self.send_response(200) - self.send_header("Content-type", ctype) - fs = os.fstat(f.fileno()) - self.send_header("Content-Length", str(fs[6])) - self.send_header("Last-Modified", self.date_time_string(fs.st_mtime)) - self.end_headers() - return f - - def list_directory(self, path): - """Helper to produce a directory listing (absent index.html). - - Return value is either a file object, or None (indicating an - error). In either case, the headers are sent, making the - interface the same as for send_head(). - - """ - try: - list = os.listdir(path) - except os.error: - self.send_error(404, "No permission to list directory") - return None - list.sort(key=lambda a: a.lower()) - r = [] - displaypath = cgi.escape(urllib.unquote(self.path)) - r.append('') - r.append("\nDirectory listing for %s\n" % displaypath) - r.append("\n

Directory listing for %s

\n" % displaypath) - r.append("
\n
    \n") - for name in list: - fullname = os.path.join(path, name) - displayname = linkname = name - # Append / for directories or @ for symbolic links - if os.path.isdir(fullname): - displayname = name + "/" - linkname = name + "/" - if os.path.islink(fullname): - displayname = name + "@" - # Note: a link to a directory displays with @ and links with / - r.append('
  • %s\n' - % (urllib.quote(linkname), cgi.escape(displayname))) - r.append("
\n
\n\n\n") - enc = sys.getfilesystemencoding() - encoded = ''.join(r).encode(enc) - f = BytesIO() - f.write(encoded) - f.seek(0) - self.send_response(200) - self.send_header("Content-type", "text/html; charset=%s" % enc) - self.send_header("Content-Length", str(len(encoded))) - self.end_headers() - return f - - def translate_path(self, path): - """Translate a /-separated PATH to the local filename syntax. - - Components that mean special things to the local file system - (e.g. drive or directory names) are ignored. (XXX They should - probably be diagnosed.) - - """ - # abandon query parameters - path = path.split('?',1)[0] - path = path.split('#',1)[0] - path = posixpath.normpath(urllib.unquote(path)) - words = path.split('/') - words = filter(None, words) - path = os.getcwd() - for word in words: - drive, word = os.path.splitdrive(word) - head, word = os.path.split(word) - if word in (os.curdir, os.pardir): continue - path = os.path.join(path, word) - return path - - def copyfile(self, source, outputfile): - """Copy all data between two file objects. - - The SOURCE argument is a file object open for reading - (or anything with a read() method) and the DESTINATION - argument is a file object open for writing (or - anything with a write() method). - - The only reason for overriding this would be to change - the block size or perhaps to replace newlines by CRLF - -- note however that this the default server uses this - to copy binary data as well. - - """ - shutil.copyfileobj(source, outputfile) - - def guess_type(self, path): - """Guess the type of a file. - - Argument is a PATH (a filename). - - Return value is a string of the form type/subtype, - usable for a MIME Content-type header. - - The default implementation looks the file's extension - up in the table self.extensions_map, using application/octet-stream - as a default; however it would be permissible (if - slow) to look inside the data to make a better guess. - - """ - - base, ext = posixpath.splitext(path) - if ext in self.extensions_map: - return self.extensions_map[ext] - ext = ext.lower() - if ext in self.extensions_map: - return self.extensions_map[ext] - else: - return self.extensions_map[''] - - if not mimetypes.inited: - mimetypes.init() # try to read system mime.types - extensions_map = mimetypes.types_map.copy() - extensions_map.update({ - '': 'application/octet-stream', # Default - '.py': 'text/plain', - '.c': 'text/plain', - '.h': 'text/plain', - }) - - -def test(HandlerClass = SimpleHTTPRequestHandler, - ServerClass = BaseHTTPServer.HTTPServer): - BaseHTTPServer.test(HandlerClass, ServerClass) - - -if __name__ == '__main__': - test() Modified: python/branches/py3k/Lib/_LWPCookieJar.py ============================================================================== --- python/branches/py3k/Lib/_LWPCookieJar.py (original) +++ python/branches/py3k/Lib/_LWPCookieJar.py Mon May 26 18:32:26 2008 @@ -12,10 +12,10 @@ """ import time, re -from cookielib import (_warn_unhandled_exception, FileCookieJar, LoadError, - Cookie, MISSING_FILENAME_TEXT, - join_header_words, split_header_words, - iso2time, time2isoz) +from http.cookiejar import (_warn_unhandled_exception, FileCookieJar, LoadError, + Cookie, MISSING_FILENAME_TEXT, + join_header_words, split_header_words, + iso2time, time2isoz) def lwp_cookie_str(cookie): """Return string representation of Cookie in an the LWP cookie file format. Modified: python/branches/py3k/Lib/_MozillaCookieJar.py ============================================================================== --- python/branches/py3k/Lib/_MozillaCookieJar.py (original) +++ python/branches/py3k/Lib/_MozillaCookieJar.py Mon May 26 18:32:26 2008 @@ -2,8 +2,8 @@ import re, time -from cookielib import (_warn_unhandled_exception, FileCookieJar, LoadError, - Cookie, MISSING_FILENAME_TEXT) +from http.cookiejar import (_warn_unhandled_exception, FileCookieJar, LoadError, + Cookie, MISSING_FILENAME_TEXT) class MozillaCookieJar(FileCookieJar): """ @@ -73,7 +73,7 @@ domain_specified = (domain_specified == "TRUE") if name == "": # cookies.txt regards 'Set-Cookie: foo' as a cookie - # with no name, whereas cookielib regards it as a + # with no name, whereas http.cookiejar regards it as a # cookie with no value. name = value value = None @@ -134,7 +134,7 @@ expires = "" if cookie.value is None: # cookies.txt regards 'Set-Cookie: foo' as a cookie - # with no name, whereas cookielib regards it as a + # with no name, whereas http.cookiejar regards it as a # cookie with no value. name = "" value = cookie.name Deleted: python/branches/py3k/Lib/cookielib.py ============================================================================== --- python/branches/py3k/Lib/cookielib.py Mon May 26 18:32:26 2008 +++ (empty file) @@ -1,1785 +0,0 @@ -"""HTTP cookie handling for web clients. - -This module has (now fairly distant) origins in Gisle Aas' Perl module -HTTP::Cookies, from the libwww-perl library. - -Docstrings, comments and debug strings in this code refer to the -attributes of the HTTP cookie system as cookie-attributes, to distinguish -them clearly from Python attributes. - -Class diagram (note that BSDDBCookieJar and the MSIE* classes are not -distributed with the Python standard library, but are available from -http://wwwsearch.sf.net/): - - CookieJar____ - / \ \ - FileCookieJar \ \ - / | \ \ \ - MozillaCookieJar | LWPCookieJar \ \ - | | \ - | ---MSIEBase | \ - | / | | \ - | / MSIEDBCookieJar BSDDBCookieJar - |/ - MSIECookieJar - -""" - -__all__ = ['Cookie', 'CookieJar', 'CookiePolicy', 'DefaultCookiePolicy', - 'FileCookieJar', 'LWPCookieJar', 'LoadError', 'MozillaCookieJar'] - -import re, urlparse, copy, time, urllib -try: - import threading as _threading -except ImportError: - import dummy_threading as _threading -import httplib # only for the default HTTP port -from calendar import timegm - -debug = False # set to True to enable debugging via the logging module -logger = None - -def _debug(*args): - if not debug: - return - global logger - if not logger: - import logging - logger = logging.getLogger("cookielib") - return logger.debug(*args) - - -DEFAULT_HTTP_PORT = str(httplib.HTTP_PORT) -MISSING_FILENAME_TEXT = ("a filename was not supplied (nor was the CookieJar " - "instance initialised with one)") - -def _warn_unhandled_exception(): - # There are a few catch-all except: statements in this module, for - # catching input that's bad in unexpected ways. Warn if any - # exceptions are caught there. - import io, warnings, traceback - f = io.StringIO() - traceback.print_exc(None, f) - msg = f.getvalue() - warnings.warn("cookielib bug!\n%s" % msg, stacklevel=2) - - -# Date/time conversion -# ----------------------------------------------------------------------------- - -EPOCH_YEAR = 1970 -def _timegm(tt): - year, month, mday, hour, min, sec = tt[:6] - if ((year >= EPOCH_YEAR) and (1 <= month <= 12) and (1 <= mday <= 31) and - (0 <= hour <= 24) and (0 <= min <= 59) and (0 <= sec <= 61)): - return timegm(tt) - else: - return None - -DAYS = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] -MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", - "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"] -MONTHS_LOWER = [] -for month in MONTHS: MONTHS_LOWER.append(month.lower()) - -def time2isoz(t=None): - """Return a string representing time in seconds since epoch, t. - - If the function is called without an argument, it will use the current - time. - - The format of the returned string is like "YYYY-MM-DD hh:mm:ssZ", - representing Universal Time (UTC, aka GMT). An example of this format is: - - 1994-11-24 08:49:37Z - - """ - if t is None: t = time.time() - year, mon, mday, hour, min, sec = time.gmtime(t)[:6] - return "%04d-%02d-%02d %02d:%02d:%02dZ" % ( - year, mon, mday, hour, min, sec) - -def time2netscape(t=None): - """Return a string representing time in seconds since epoch, t. - - If the function is called without an argument, it will use the current - time. - - The format of the returned string is like this: - - Wed, DD-Mon-YYYY HH:MM:SS GMT - - """ - if t is None: t = time.time() - year, mon, mday, hour, min, sec, wday = time.gmtime(t)[:7] - return "%s %02d-%s-%04d %02d:%02d:%02d GMT" % ( - DAYS[wday], mday, MONTHS[mon-1], year, hour, min, sec) - - -UTC_ZONES = {"GMT": None, "UTC": None, "UT": None, "Z": None} - -TIMEZONE_RE = re.compile(r"^([-+])?(\d\d?):?(\d\d)?$") -def offset_from_tz_string(tz): - offset = None - if tz in UTC_ZONES: - offset = 0 - else: - m = TIMEZONE_RE.search(tz) - if m: - offset = 3600 * int(m.group(2)) - if m.group(3): - offset = offset + 60 * int(m.group(3)) - if m.group(1) == '-': - offset = -offset - return offset - -def _str2time(day, mon, yr, hr, min, sec, tz): - # translate month name to number - # month numbers start with 1 (January) - try: - mon = MONTHS_LOWER.index(mon.lower())+1 - except ValueError: - # maybe it's already a number - try: - imon = int(mon) - except ValueError: - return None - if 1 <= imon <= 12: - mon = imon - else: - return None - - # make sure clock elements are defined - if hr is None: hr = 0 - if min is None: min = 0 - if sec is None: sec = 0 - - yr = int(yr) - day = int(day) - hr = int(hr) - min = int(min) - sec = int(sec) - - if yr < 1000: - # find "obvious" year - cur_yr = time.localtime(time.time())[0] - m = cur_yr % 100 - tmp = yr - yr = yr + cur_yr - m - m = m - tmp - if abs(m) > 50: - if m > 0: yr = yr + 100 - else: yr = yr - 100 - - # convert UTC time tuple to seconds since epoch (not timezone-adjusted) - t = _timegm((yr, mon, day, hr, min, sec, tz)) - - if t is not None: - # adjust time using timezone string, to get absolute time since epoch - if tz is None: - tz = "UTC" - tz = tz.upper() - offset = offset_from_tz_string(tz) - if offset is None: - return None - t = t - offset - - return t - -STRICT_DATE_RE = re.compile( - r"^[SMTWF][a-z][a-z], (\d\d) ([JFMASOND][a-z][a-z]) " - "(\d\d\d\d) (\d\d):(\d\d):(\d\d) GMT$") -WEEKDAY_RE = re.compile( - r"^(?:Sun|Mon|Tue|Wed|Thu|Fri|Sat)[a-z]*,?\s*", re.I) -LOOSE_HTTP_DATE_RE = re.compile( - r"""^ - (\d\d?) # day - (?:\s+|[-\/]) - (\w+) # month - (?:\s+|[-\/]) - (\d+) # year - (?: - (?:\s+|:) # separator before clock - (\d\d?):(\d\d) # hour:min - (?::(\d\d))? # optional seconds - )? # optional clock - \s* - ([-+]?\d{2,4}|(?![APap][Mm]\b)[A-Za-z]+)? # timezone - \s* - (?:\(\w+\))? # ASCII representation of timezone in parens. - \s*$""", re.X) -def http2time(text): - """Returns time in seconds since epoch of time represented by a string. - - Return value is an integer. - - None is returned if the format of str is unrecognized, the time is outside - the representable range, or the timezone string is not recognized. If the - string contains no timezone, UTC is assumed. - - The timezone in the string may be numerical (like "-0800" or "+0100") or a - string timezone (like "UTC", "GMT", "BST" or "EST"). Currently, only the - timezone strings equivalent to UTC (zero offset) are known to the function. - - The function loosely parses the following formats: - - Wed, 09 Feb 1994 22:23:32 GMT -- HTTP format - Tuesday, 08-Feb-94 14:15:29 GMT -- old rfc850 HTTP format - Tuesday, 08-Feb-1994 14:15:29 GMT -- broken rfc850 HTTP format - 09 Feb 1994 22:23:32 GMT -- HTTP format (no weekday) - 08-Feb-94 14:15:29 GMT -- rfc850 format (no weekday) - 08-Feb-1994 14:15:29 GMT -- broken rfc850 format (no weekday) - - The parser ignores leading and trailing whitespace. The time may be - absent. - - If the year is given with only 2 digits, the function will select the - century that makes the year closest to the current date. - - """ - # fast exit for strictly conforming string - m = STRICT_DATE_RE.search(text) - if m: - g = m.groups() - mon = MONTHS_LOWER.index(g[1].lower()) + 1 - tt = (int(g[2]), mon, int(g[0]), - int(g[3]), int(g[4]), float(g[5])) - return _timegm(tt) - - # No, we need some messy parsing... - - # clean up - text = text.lstrip() - text = WEEKDAY_RE.sub("", text, 1) # Useless weekday - - # tz is time zone specifier string - day, mon, yr, hr, min, sec, tz = [None]*7 - - # loose regexp parse - m = LOOSE_HTTP_DATE_RE.search(text) - if m is not None: - day, mon, yr, hr, min, sec, tz = m.groups() - else: - return None # bad format - - return _str2time(day, mon, yr, hr, min, sec, tz) - -ISO_DATE_RE = re.compile( - """^ - (\d{4}) # year - [-\/]? - (\d\d?) # numerical month - [-\/]? - (\d\d?) # day - (?: - (?:\s+|[-:Tt]) # separator before clock - (\d\d?):?(\d\d) # hour:min - (?::?(\d\d(?:\.\d*)?))? # optional seconds (and fractional) - )? # optional clock - \s* - ([-+]?\d\d?:?(:?\d\d)? - |Z|z)? # timezone (Z is "zero meridian", i.e. GMT) - \s*$""", re.X) -def iso2time(text): - """ - As for http2time, but parses the ISO 8601 formats: - - 1994-02-03 14:15:29 -0100 -- ISO 8601 format - 1994-02-03 14:15:29 -- zone is optional - 1994-02-03 -- only date - 1994-02-03T14:15:29 -- Use T as separator - 19940203T141529Z -- ISO 8601 compact format - 19940203 -- only date - - """ - # clean up - text = text.lstrip() - - # tz is time zone specifier string - day, mon, yr, hr, min, sec, tz = [None]*7 - - # loose regexp parse - m = ISO_DATE_RE.search(text) - if m is not None: - # XXX there's an extra bit of the timezone I'm ignoring here: is - # this the right thing to do? - yr, mon, day, hr, min, sec, tz, _ = m.groups() - else: - return None # bad format - - return _str2time(day, mon, yr, hr, min, sec, tz) - - -# Header parsing -# ----------------------------------------------------------------------------- - -def unmatched(match): - """Return unmatched part of re.Match object.""" - start, end = match.span(0) - return match.string[:start]+match.string[end:] - -HEADER_TOKEN_RE = re.compile(r"^\s*([^=\s;,]+)") -HEADER_QUOTED_VALUE_RE = re.compile(r"^\s*=\s*\"([^\"\\]*(?:\\.[^\"\\]*)*)\"") -HEADER_VALUE_RE = re.compile(r"^\s*=\s*([^\s;,]*)") -HEADER_ESCAPE_RE = re.compile(r"\\(.)") -def split_header_words(header_values): - r"""Parse header values into a list of lists containing key,value pairs. - - The function knows how to deal with ",", ";" and "=" as well as quoted - values after "=". A list of space separated tokens are parsed as if they - were separated by ";". - - If the header_values passed as argument contains multiple values, then they - are treated as if they were a single value separated by comma ",". - - This means that this function is useful for parsing header fields that - follow this syntax (BNF as from the HTTP/1.1 specification, but we relax - the requirement for tokens). - - headers = #header - header = (token | parameter) *( [";"] (token | parameter)) - - token = 1* - separators = "(" | ")" | "<" | ">" | "@" - | "," | ";" | ":" | "\" | <"> - | "/" | "[" | "]" | "?" | "=" - | "{" | "}" | SP | HT - - quoted-string = ( <"> *(qdtext | quoted-pair ) <"> ) - qdtext = > - quoted-pair = "\" CHAR - - parameter = attribute "=" value - attribute = token - value = token | quoted-string - - Each header is represented by a list of key/value pairs. The value for a - simple token (not part of a parameter) is None. Syntactically incorrect - headers will not necessarily be parsed as you would want. - - This is easier to describe with some examples: - - >>> split_header_words(['foo="bar"; port="80,81"; discard, bar=baz']) - [[('foo', 'bar'), ('port', '80,81'), ('discard', None)], [('bar', 'baz')]] - >>> split_header_words(['text/html; charset="iso-8859-1"']) - [[('text/html', None), ('charset', 'iso-8859-1')]] - >>> split_header_words([r'Basic realm="\"foo\bar\""']) - [[('Basic', None), ('realm', '"foobar"')]] - - """ - assert not isinstance(header_values, str) - result = [] - for text in header_values: - orig_text = text - pairs = [] - while text: - m = HEADER_TOKEN_RE.search(text) - if m: - text = unmatched(m) - name = m.group(1) - m = HEADER_QUOTED_VALUE_RE.search(text) - if m: # quoted value - text = unmatched(m) - value = m.group(1) - value = HEADER_ESCAPE_RE.sub(r"\1", value) - else: - m = HEADER_VALUE_RE.search(text) - if m: # unquoted value - text = unmatched(m) - value = m.group(1) - value = value.rstrip() - else: - # no value, a lone token - value = None - pairs.append((name, value)) - elif text.lstrip().startswith(","): - # concatenated headers, as per RFC 2616 section 4.2 - text = text.lstrip()[1:] - if pairs: result.append(pairs) - pairs = [] - else: - # skip junk - non_junk, nr_junk_chars = re.subn("^[=\s;]*", "", text) - assert nr_junk_chars > 0, ( - "split_header_words bug: '%s', '%s', %s" % - (orig_text, text, pairs)) - text = non_junk - if pairs: result.append(pairs) - return result - -HEADER_JOIN_ESCAPE_RE = re.compile(r"([\"\\])") -def join_header_words(lists): - """Do the inverse (almost) of the conversion done by split_header_words. - - Takes a list of lists of (key, value) pairs and produces a single header - value. Attribute values are quoted if needed. - - >>> join_header_words([[("text/plain", None), ("charset", "iso-8859/1")]]) - 'text/plain; charset="iso-8859/1"' - >>> join_header_words([[("text/plain", None)], [("charset", "iso-8859/1")]]) - 'text/plain, charset="iso-8859/1"' - - """ - headers = [] - for pairs in lists: - attr = [] - for k, v in pairs: - if v is not None: - if not re.search(r"^\w+$", v): - v = HEADER_JOIN_ESCAPE_RE.sub(r"\\\1", v) # escape " and \ - v = '"%s"' % v - k = "%s=%s" % (k, v) - attr.append(k) - if attr: headers.append("; ".join(attr)) - return ", ".join(headers) - -def parse_ns_headers(ns_headers): - """Ad-hoc parser for Netscape protocol cookie-attributes. - - The old Netscape cookie format for Set-Cookie can for instance contain - an unquoted "," in the expires field, so we have to use this ad-hoc - parser instead of split_header_words. - - XXX This may not make the best possible effort to parse all the crap - that Netscape Cookie headers contain. Ronald Tschalar's HTTPClient - parser is probably better, so could do worse than following that if - this ever gives any trouble. - - Currently, this is also used for parsing RFC 2109 cookies. - - """ - known_attrs = ("expires", "domain", "path", "secure", - # RFC 2109 attrs (may turn up in Netscape cookies, too) - "port", "max-age") - - result = [] - for ns_header in ns_headers: - pairs = [] - version_set = False - for ii, param in enumerate(re.split(r";\s*", ns_header)): - param = param.rstrip() - if param == "": continue - if "=" not in param: - k, v = param, None - else: - k, v = re.split(r"\s*=\s*", param, 1) - k = k.lstrip() - if ii != 0: - lc = k.lower() - if lc in known_attrs: - k = lc - if k == "version": - # This is an RFC 2109 cookie. - version_set = True - if k == "expires": - # convert expires date to seconds since epoch - if v.startswith('"'): v = v[1:] - if v.endswith('"'): v = v[:-1] - v = http2time(v) # None if invalid - pairs.append((k, v)) - - if pairs: - if not version_set: - pairs.append(("version", "0")) - result.append(pairs) - - return result - - -IPV4_RE = re.compile(r"\.\d+$") -def is_HDN(text): - """Return True if text is a host domain name.""" - # XXX - # This may well be wrong. Which RFC is HDN defined in, if any (for - # the purposes of RFC 2965)? - # For the current implementation, what about IPv6? Remember to look - # at other uses of IPV4_RE also, if change this. - if IPV4_RE.search(text): - return False - if text == "": - return False - if text[0] == "." or text[-1] == ".": - return False - return True - -def domain_match(A, B): - """Return True if domain A domain-matches domain B, according to RFC 2965. - - A and B may be host domain names or IP addresses. - - RFC 2965, section 1: - - Host names can be specified either as an IP address or a HDN string. - Sometimes we compare one host name with another. (Such comparisons SHALL - be case-insensitive.) Host A's name domain-matches host B's if - - * their host name strings string-compare equal; or - - * A is a HDN string and has the form NB, where N is a non-empty - name string, B has the form .B', and B' is a HDN string. (So, - x.y.com domain-matches .Y.com but not Y.com.) - - Note that domain-match is not a commutative operation: a.b.c.com - domain-matches .c.com, but not the reverse. - - """ - # Note that, if A or B are IP addresses, the only relevant part of the - # definition of the domain-match algorithm is the direct string-compare. - A = A.lower() - B = B.lower() - if A == B: - return True - if not is_HDN(A): - return False - i = A.rfind(B) - if i == -1 or i == 0: - # A does not have form NB, or N is the empty string - return False - if not B.startswith("."): - return False - if not is_HDN(B[1:]): - return False - return True - -def liberal_is_HDN(text): - """Return True if text is a sort-of-like a host domain name. - - For accepting/blocking domains. - - """ - if IPV4_RE.search(text): - return False - return True - -def user_domain_match(A, B): - """For blocking/accepting domains. - - A and B may be host domain names or IP addresses. - - """ - A = A.lower() - B = B.lower() - if not (liberal_is_HDN(A) and liberal_is_HDN(B)): - if A == B: - # equal IP addresses - return True - return False - initial_dot = B.startswith(".") - if initial_dot and A.endswith(B): - return True - if not initial_dot and A == B: - return True - return False - -cut_port_re = re.compile(r":\d+$") -def request_host(request): - """Return request-host, as defined by RFC 2965. - - Variation from RFC: returned value is lowercased, for convenient - comparison. - - """ - url = request.get_full_url() - host = urlparse.urlparse(url)[1] - if host == "": - host = request.get_header("Host", "") - - # remove port, if present - host = cut_port_re.sub("", host, 1) - return host.lower() - -def eff_request_host(request): - """Return a tuple (request-host, effective request-host name). - - As defined by RFC 2965, except both are lowercased. - - """ - erhn = req_host = request_host(request) - if req_host.find(".") == -1 and not IPV4_RE.search(req_host): - erhn = req_host + ".local" - return req_host, erhn - -def request_path(request): - """request-URI, as defined by RFC 2965.""" - url = request.get_full_url() - #scheme, netloc, path, parameters, query, frag = urlparse.urlparse(url) - #req_path = escape_path("".join(urlparse.urlparse(url)[2:])) - path, parameters, query, frag = urlparse.urlparse(url)[2:] - if parameters: - path = "%s;%s" % (path, parameters) - path = escape_path(path) - req_path = urlparse.urlunparse(("", "", path, "", query, frag)) - if not req_path.startswith("/"): - # fix bad RFC 2396 absoluteURI - req_path = "/"+req_path - return req_path - -def request_port(request): - host = request.get_host() - i = host.find(':') - if i >= 0: - port = host[i+1:] - try: - int(port) - except ValueError: - _debug("nonnumeric port: '%s'", port) - return None - else: - port = DEFAULT_HTTP_PORT - return port - -# Characters in addition to A-Z, a-z, 0-9, '_', '.', and '-' that don't -# need to be escaped to form a valid HTTP URL (RFCs 2396 and 1738). -HTTP_PATH_SAFE = "%/;:@&=+$,!~*'()" -ESCAPED_CHAR_RE = re.compile(r"%([0-9a-fA-F][0-9a-fA-F])") -def uppercase_escaped_char(match): - return "%%%s" % match.group(1).upper() -def escape_path(path): - """Escape any invalid characters in HTTP URL, and uppercase all escapes.""" - # There's no knowing what character encoding was used to create URLs - # containing %-escapes, but since we have to pick one to escape invalid - # path characters, we pick UTF-8, as recommended in the HTML 4.0 - # specification: - # http://www.w3.org/TR/REC-html40/appendix/notes.html#h-B.2.1 - # And here, kind of: draft-fielding-uri-rfc2396bis-03 - # (And in draft IRI specification: draft-duerst-iri-05) - # (And here, for new URI schemes: RFC 2718) - path = urllib.quote(path, HTTP_PATH_SAFE) - path = ESCAPED_CHAR_RE.sub(uppercase_escaped_char, path) - return path - -def reach(h): - """Return reach of host h, as defined by RFC 2965, section 1. - - The reach R of a host name H is defined as follows: - - * If - - - H is the host domain name of a host; and, - - - H has the form A.B; and - - - A has no embedded (that is, interior) dots; and - - - B has at least one embedded dot, or B is the string "local". - then the reach of H is .B. - - * Otherwise, the reach of H is H. - - >>> reach("www.acme.com") - '.acme.com' - >>> reach("acme.com") - 'acme.com' - >>> reach("acme.local") - '.local' - - """ - i = h.find(".") - if i >= 0: - #a = h[:i] # this line is only here to show what a is - b = h[i+1:] - i = b.find(".") - if is_HDN(h) and (i >= 0 or b == "local"): - return "."+b - return h - -def is_third_party(request): - """ - - RFC 2965, section 3.3.6: - - An unverifiable transaction is to a third-party host if its request- - host U does not domain-match the reach R of the request-host O in the - origin transaction. - - """ - req_host = request_host(request) - if not domain_match(req_host, reach(request.get_origin_req_host())): - return True - else: - return False - - -class Cookie: - """HTTP Cookie. - - This class represents both Netscape and RFC 2965 cookies. - - This is deliberately a very simple class. It just holds attributes. It's - possible to construct Cookie instances that don't comply with the cookie - standards. CookieJar.make_cookies is the factory function for Cookie - objects -- it deals with cookie parsing, supplying defaults, and - normalising to the representation used in this class. CookiePolicy is - responsible for checking them to see whether they should be accepted from - and returned to the server. - - Note that the port may be present in the headers, but unspecified ("Port" - rather than"Port=80", for example); if this is the case, port is None. - - """ - - def __init__(self, version, name, value, - port, port_specified, - domain, domain_specified, domain_initial_dot, - path, path_specified, - secure, - expires, - discard, - comment, - comment_url, - rest, - rfc2109=False, - ): - - if version is not None: version = int(version) - if expires is not None: expires = int(expires) - if port is None and port_specified is True: - raise ValueError("if port is None, port_specified must be false") - - self.version = version - self.name = name - self.value = value - self.port = port - self.port_specified = port_specified - # normalise case, as per RFC 2965 section 3.3.3 - self.domain = domain.lower() - self.domain_specified = domain_specified - # Sigh. We need to know whether the domain given in the - # cookie-attribute had an initial dot, in order to follow RFC 2965 - # (as clarified in draft errata). Needed for the returned $Domain - # value. - self.domain_initial_dot = domain_initial_dot - self.path = path - self.path_specified = path_specified - self.secure = secure - self.expires = expires - self.discard = discard - self.comment = comment - self.comment_url = comment_url - self.rfc2109 = rfc2109 - - self._rest = copy.copy(rest) - - def has_nonstandard_attr(self, name): - return name in self._rest - def get_nonstandard_attr(self, name, default=None): - return self._rest.get(name, default) - def set_nonstandard_attr(self, name, value): - self._rest[name] = value - - def is_expired(self, now=None): - if now is None: now = time.time() - if (self.expires is not None) and (self.expires <= now): - return True - return False - - def __str__(self): - if self.port is None: p = "" - else: p = ":"+self.port - limit = self.domain + p + self.path - if self.value is not None: - namevalue = "%s=%s" % (self.name, self.value) - else: - namevalue = self.name - return "" % (namevalue, limit) - - def __repr__(self): - args = [] - for name in ("version", "name", "value", - "port", "port_specified", - "domain", "domain_specified", "domain_initial_dot", - "path", "path_specified", - "secure", "expires", "discard", "comment", "comment_url", - ): - attr = getattr(self, name) - args.append("%s=%s" % (name, repr(attr))) - args.append("rest=%s" % repr(self._rest)) - args.append("rfc2109=%s" % repr(self.rfc2109)) - return "Cookie(%s)" % ", ".join(args) - - -class CookiePolicy: - """Defines which cookies get accepted from and returned to server. - - May also modify cookies, though this is probably a bad idea. - - The subclass DefaultCookiePolicy defines the standard rules for Netscape - and RFC 2965 cookies -- override that if you want a customised policy. - - """ - def set_ok(self, cookie, request): - """Return true if (and only if) cookie should be accepted from server. - - Currently, pre-expired cookies never get this far -- the CookieJar - class deletes such cookies itself. - - """ - raise NotImplementedError() - - def return_ok(self, cookie, request): - """Return true if (and only if) cookie should be returned to server.""" - raise NotImplementedError() - - def domain_return_ok(self, domain, request): - """Return false if cookies should not be returned, given cookie domain. - """ - return True - - def path_return_ok(self, path, request): - """Return false if cookies should not be returned, given cookie path. - """ - return True - - -class DefaultCookiePolicy(CookiePolicy): - """Implements the standard rules for accepting and returning cookies.""" - - DomainStrictNoDots = 1 - DomainStrictNonDomain = 2 - DomainRFC2965Match = 4 - - DomainLiberal = 0 - DomainStrict = DomainStrictNoDots|DomainStrictNonDomain - - def __init__(self, - blocked_domains=None, allowed_domains=None, - netscape=True, rfc2965=False, - rfc2109_as_netscape=None, - hide_cookie2=False, - strict_domain=False, - strict_rfc2965_unverifiable=True, - strict_ns_unverifiable=False, - strict_ns_domain=DomainLiberal, - strict_ns_set_initial_dollar=False, - strict_ns_set_path=False, - ): - """Constructor arguments should be passed as keyword arguments only.""" - self.netscape = netscape - self.rfc2965 = rfc2965 - self.rfc2109_as_netscape = rfc2109_as_netscape - self.hide_cookie2 = hide_cookie2 - self.strict_domain = strict_domain - self.strict_rfc2965_unverifiable = strict_rfc2965_unverifiable - self.strict_ns_unverifiable = strict_ns_unverifiable - self.strict_ns_domain = strict_ns_domain - self.strict_ns_set_initial_dollar = strict_ns_set_initial_dollar - self.strict_ns_set_path = strict_ns_set_path - - if blocked_domains is not None: - self._blocked_domains = tuple(blocked_domains) - else: - self._blocked_domains = () - - if allowed_domains is not None: - allowed_domains = tuple(allowed_domains) - self._allowed_domains = allowed_domains - - def blocked_domains(self): - """Return the sequence of blocked domains (as a tuple).""" - return self._blocked_domains - def set_blocked_domains(self, blocked_domains): - """Set the sequence of blocked domains.""" - self._blocked_domains = tuple(blocked_domains) - - def is_blocked(self, domain): - for blocked_domain in self._blocked_domains: - if user_domain_match(domain, blocked_domain): - return True - return False - - def allowed_domains(self): - """Return None, or the sequence of allowed domains (as a tuple).""" - return self._allowed_domains - def set_allowed_domains(self, allowed_domains): - """Set the sequence of allowed domains, or None.""" - if allowed_domains is not None: - allowed_domains = tuple(allowed_domains) - self._allowed_domains = allowed_domains - - def is_not_allowed(self, domain): - if self._allowed_domains is None: - return False - for allowed_domain in self._allowed_domains: - if user_domain_match(domain, allowed_domain): - return False - return True - - def set_ok(self, cookie, request): - """ - If you override .set_ok(), be sure to call this method. If it returns - false, so should your subclass (assuming your subclass wants to be more - strict about which cookies to accept). - - """ - _debug(" - checking cookie %s=%s", cookie.name, cookie.value) - - assert cookie.name is not None - - for n in "version", "verifiability", "name", "path", "domain", "port": - fn_name = "set_ok_"+n - fn = getattr(self, fn_name) - if not fn(cookie, request): - return False - - return True - - def set_ok_version(self, cookie, request): - if cookie.version is None: - # Version is always set to 0 by parse_ns_headers if it's a Netscape - # cookie, so this must be an invalid RFC 2965 cookie. - _debug(" Set-Cookie2 without version attribute (%s=%s)", - cookie.name, cookie.value) - return False - if cookie.version > 0 and not self.rfc2965: - _debug(" RFC 2965 cookies are switched off") - return False - elif cookie.version == 0 and not self.netscape: - _debug(" Netscape cookies are switched off") - return False - return True - - def set_ok_verifiability(self, cookie, request): - if request.is_unverifiable() and is_third_party(request): - if cookie.version > 0 and self.strict_rfc2965_unverifiable: - _debug(" third-party RFC 2965 cookie during " - "unverifiable transaction") - return False - elif cookie.version == 0 and self.strict_ns_unverifiable: - _debug(" third-party Netscape cookie during " - "unverifiable transaction") - return False - return True - - def set_ok_name(self, cookie, request): - # Try and stop servers setting V0 cookies designed to hack other - # servers that know both V0 and V1 protocols. - if (cookie.version == 0 and self.strict_ns_set_initial_dollar and - cookie.name.startswith("$")): - _debug(" illegal name (starts with '$'): '%s'", cookie.name) - return False - return True - - def set_ok_path(self, cookie, request): - if cookie.path_specified: - req_path = request_path(request) - if ((cookie.version > 0 or - (cookie.version == 0 and self.strict_ns_set_path)) and - not req_path.startswith(cookie.path)): - _debug(" path attribute %s is not a prefix of request " - "path %s", cookie.path, req_path) - return False - return True - - def set_ok_domain(self, cookie, request): - if self.is_blocked(cookie.domain): - _debug(" domain %s is in user block-list", cookie.domain) - return False - if self.is_not_allowed(cookie.domain): - _debug(" domain %s is not in user allow-list", cookie.domain) - return False - if cookie.domain_specified: - req_host, erhn = eff_request_host(request) - domain = cookie.domain - if self.strict_domain and (domain.count(".") >= 2): - # XXX This should probably be compared with the Konqueror - # (kcookiejar.cpp) and Mozilla implementations, but it's a - # losing battle. - i = domain.rfind(".") - j = domain.rfind(".", 0, i) - if j == 0: # domain like .foo.bar - tld = domain[i+1:] - sld = domain[j+1:i] - if sld.lower() in ("co", "ac", "com", "edu", "org", "net", - "gov", "mil", "int", "aero", "biz", "cat", "coop", - "info", "jobs", "mobi", "museum", "name", "pro", - "travel", "eu") and len(tld) == 2: - # domain like .co.uk - _debug(" country-code second level domain %s", domain) - return False - if domain.startswith("."): - undotted_domain = domain[1:] - else: - undotted_domain = domain - embedded_dots = (undotted_domain.find(".") >= 0) - if not embedded_dots and domain != ".local": - _debug(" non-local domain %s contains no embedded dot", - domain) - return False - if cookie.version == 0: - if (not erhn.endswith(domain) and - (not erhn.startswith(".") and - not ("."+erhn).endswith(domain))): - _debug(" effective request-host %s (even with added " - "initial dot) does not end end with %s", - erhn, domain) - return False - if (cookie.version > 0 or - (self.strict_ns_domain & self.DomainRFC2965Match)): - if not domain_match(erhn, domain): - _debug(" effective request-host %s does not domain-match " - "%s", erhn, domain) - return False - if (cookie.version > 0 or - (self.strict_ns_domain & self.DomainStrictNoDots)): - host_prefix = req_host[:-len(domain)] - if (host_prefix.find(".") >= 0 and - not IPV4_RE.search(req_host)): - _debug(" host prefix %s for domain %s contains a dot", - host_prefix, domain) - return False - return True - - def set_ok_port(self, cookie, request): - if cookie.port_specified: - req_port = request_port(request) - if req_port is None: - req_port = "80" - else: - req_port = str(req_port) - for p in cookie.port.split(","): - try: - int(p) - except ValueError: - _debug(" bad port %s (not numeric)", p) - return False - if p == req_port: - break - else: - _debug(" request port (%s) not found in %s", - req_port, cookie.port) - return False - return True - - def return_ok(self, cookie, request): - """ - If you override .return_ok(), be sure to call this method. If it - returns false, so should your subclass (assuming your subclass wants to - be more strict about which cookies to return). - - """ - # Path has already been checked by .path_return_ok(), and domain - # blocking done by .domain_return_ok(). - _debug(" - checking cookie %s=%s", cookie.name, cookie.value) - - for n in "version", "verifiability", "secure", "expires", "port", "domain": - fn_name = "return_ok_"+n - fn = getattr(self, fn_name) - if not fn(cookie, request): - return False - return True - - def return_ok_version(self, cookie, request): - if cookie.version > 0 and not self.rfc2965: - _debug(" RFC 2965 cookies are switched off") - return False - elif cookie.version == 0 and not self.netscape: - _debug(" Netscape cookies are switched off") - return False - return True - - def return_ok_verifiability(self, cookie, request): - if request.is_unverifiable() and is_third_party(request): - if cookie.version > 0 and self.strict_rfc2965_unverifiable: - _debug(" third-party RFC 2965 cookie during unverifiable " - "transaction") - return False - elif cookie.version == 0 and self.strict_ns_unverifiable: - _debug(" third-party Netscape cookie during unverifiable " - "transaction") - return False - return True - - def return_ok_secure(self, cookie, request): - if cookie.secure and request.get_type() != "https": - _debug(" secure cookie with non-secure request") - return False - return True - - def return_ok_expires(self, cookie, request): - if cookie.is_expired(self._now): - _debug(" cookie expired") - return False - return True - - def return_ok_port(self, cookie, request): - if cookie.port: - req_port = request_port(request) - if req_port is None: - req_port = "80" - for p in cookie.port.split(","): - if p == req_port: - break - else: - _debug(" request port %s does not match cookie port %s", - req_port, cookie.port) - return False - return True - - def return_ok_domain(self, cookie, request): - req_host, erhn = eff_request_host(request) - domain = cookie.domain - - # strict check of non-domain cookies: Mozilla does this, MSIE5 doesn't - if (cookie.version == 0 and - (self.strict_ns_domain & self.DomainStrictNonDomain) and - not cookie.domain_specified and domain != erhn): - _debug(" cookie with unspecified domain does not string-compare " - "equal to request domain") - return False - - if cookie.version > 0 and not domain_match(erhn, domain): - _debug(" effective request-host name %s does not domain-match " - "RFC 2965 cookie domain %s", erhn, domain) - return False - if cookie.version == 0 and not ("."+erhn).endswith(domain): - _debug(" request-host %s does not match Netscape cookie domain " - "%s", req_host, domain) - return False - return True - - def domain_return_ok(self, domain, request): - # Liberal check of. This is here as an optimization to avoid - # having to load lots of MSIE cookie files unless necessary. - req_host, erhn = eff_request_host(request) - if not req_host.startswith("."): - req_host = "."+req_host - if not erhn.startswith("."): - erhn = "."+erhn - if not (req_host.endswith(domain) or erhn.endswith(domain)): - #_debug(" request domain %s does not match cookie domain %s", - # req_host, domain) - return False - - if self.is_blocked(domain): - _debug(" domain %s is in user block-list", domain) - return False - if self.is_not_allowed(domain): - _debug(" domain %s is not in user allow-list", domain) - return False - - return True - - def path_return_ok(self, path, request): - _debug("- checking cookie path=%s", path) - req_path = request_path(request) - if not req_path.startswith(path): - _debug(" %s does not path-match %s", req_path, path) - return False - return True - - -def vals_sorted_by_key(adict): - keys = sorted(adict.keys()) - return map(adict.get, keys) - -def deepvalues(mapping): - """Iterates over nested mapping, depth-first, in sorted order by key.""" - values = vals_sorted_by_key(mapping) - for obj in values: - mapping = False - try: - obj.items - except AttributeError: - pass - else: - mapping = True - for subobj in deepvalues(obj): - yield subobj - if not mapping: - yield obj - - -# Used as second parameter to dict.get() method, to distinguish absent -# dict key from one with a None value. -class Absent: pass - -class CookieJar: - """Collection of HTTP cookies. - - You may not need to know about this class: try - urllib2.build_opener(HTTPCookieProcessor).open(url). - - """ - - non_word_re = re.compile(r"\W") - quote_re = re.compile(r"([\"\\])") - strict_domain_re = re.compile(r"\.?[^.]*") - domain_re = re.compile(r"[^.]*") - dots_re = re.compile(r"^\.+") - - magic_re = r"^\#LWP-Cookies-(\d+\.\d+)" - - def __init__(self, policy=None): - if policy is None: - policy = DefaultCookiePolicy() - self._policy = policy - - self._cookies_lock = _threading.RLock() - self._cookies = {} - - def set_policy(self, policy): - self._policy = policy - - def _cookies_for_domain(self, domain, request): - cookies = [] - if not self._policy.domain_return_ok(domain, request): - return [] - _debug("Checking %s for cookies to return", domain) - cookies_by_path = self._cookies[domain] - for path in cookies_by_path.keys(): - if not self._policy.path_return_ok(path, request): - continue - cookies_by_name = cookies_by_path[path] - for cookie in cookies_by_name.values(): - if not self._policy.return_ok(cookie, request): - _debug(" not returning cookie") - continue - _debug(" it's a match") - cookies.append(cookie) - return cookies - - def _cookies_for_request(self, request): - """Return a list of cookies to be returned to server.""" - cookies = [] - for domain in self._cookies.keys(): - cookies.extend(self._cookies_for_domain(domain, request)) - return cookies - - def _cookie_attrs(self, cookies): - """Return a list of cookie-attributes to be returned to server. - - like ['foo="bar"; $Path="/"', ...] - - The $Version attribute is also added when appropriate (currently only - once per request). - - """ - # add cookies in order of most specific (ie. longest) path first - cookies.sort(key=lambda a: len(a.path), reverse=True) - - version_set = False - - attrs = [] - for cookie in cookies: - # set version of Cookie header - # XXX - # What should it be if multiple matching Set-Cookie headers have - # different versions themselves? - # Answer: there is no answer; was supposed to be settled by - # RFC 2965 errata, but that may never appear... - version = cookie.version - if not version_set: - version_set = True - if version > 0: - attrs.append("$Version=%s" % version) - - # quote cookie value if necessary - # (not for Netscape protocol, which already has any quotes - # intact, due to the poorly-specified Netscape Cookie: syntax) - if ((cookie.value is not None) and - self.non_word_re.search(cookie.value) and version > 0): - value = self.quote_re.sub(r"\\\1", cookie.value) - else: - value = cookie.value - - # add cookie-attributes to be returned in Cookie header - if cookie.value is None: - attrs.append(cookie.name) - else: - attrs.append("%s=%s" % (cookie.name, value)) - if version > 0: - if cookie.path_specified: - attrs.append('$Path="%s"' % cookie.path) - if cookie.domain.startswith("."): - domain = cookie.domain - if (not cookie.domain_initial_dot and - domain.startswith(".")): - domain = domain[1:] - attrs.append('$Domain="%s"' % domain) - if cookie.port is not None: - p = "$Port" - if cookie.port_specified: - p = p + ('="%s"' % cookie.port) - attrs.append(p) - - return attrs - - def add_cookie_header(self, request): - """Add correct Cookie: header to request (urllib2.Request object). - - The Cookie2 header is also added unless policy.hide_cookie2 is true. - - """ - _debug("add_cookie_header") - self._cookies_lock.acquire() - try: - - self._policy._now = self._now = int(time.time()) - - cookies = self._cookies_for_request(request) - - attrs = self._cookie_attrs(cookies) - if attrs: - if not request.has_header("Cookie"): - request.add_unredirected_header( - "Cookie", "; ".join(attrs)) - - # if necessary, advertise that we know RFC 2965 - if (self._policy.rfc2965 and not self._policy.hide_cookie2 and - not request.has_header("Cookie2")): - for cookie in cookies: - if cookie.version != 1: - request.add_unredirected_header("Cookie2", '$Version="1"') - break - - finally: - self._cookies_lock.release() - - self.clear_expired_cookies() - - def _normalized_cookie_tuples(self, attrs_set): - """Return list of tuples containing normalised cookie information. - - attrs_set is the list of lists of key,value pairs extracted from - the Set-Cookie or Set-Cookie2 headers. - - Tuples are name, value, standard, rest, where name and value are the - cookie name and value, standard is a dictionary containing the standard - cookie-attributes (discard, secure, version, expires or max-age, - domain, path and port) and rest is a dictionary containing the rest of - the cookie-attributes. - - """ - cookie_tuples = [] - - boolean_attrs = "discard", "secure" - value_attrs = ("version", - "expires", "max-age", - "domain", "path", "port", - "comment", "commenturl") - - for cookie_attrs in attrs_set: - name, value = cookie_attrs[0] - - # Build dictionary of standard cookie-attributes (standard) and - # dictionary of other cookie-attributes (rest). - - # Note: expiry time is normalised to seconds since epoch. V0 - # cookies should have the Expires cookie-attribute, and V1 cookies - # should have Max-Age, but since V1 includes RFC 2109 cookies (and - # since V0 cookies may be a mish-mash of Netscape and RFC 2109), we - # accept either (but prefer Max-Age). - max_age_set = False - - bad_cookie = False - - standard = {} - rest = {} - for k, v in cookie_attrs[1:]: - lc = k.lower() - # don't lose case distinction for unknown fields - if lc in value_attrs or lc in boolean_attrs: - k = lc - if k in boolean_attrs and v is None: - # boolean cookie-attribute is present, but has no value - # (like "discard", rather than "port=80") - v = True - if k in standard: - # only first value is significant - continue - if k == "domain": - if v is None: - _debug(" missing value for domain attribute") - bad_cookie = True - break - # RFC 2965 section 3.3.3 - v = v.lower() - if k == "expires": - if max_age_set: - # Prefer max-age to expires (like Mozilla) - continue - if v is None: - _debug(" missing or invalid value for expires " - "attribute: treating as session cookie") - continue - if k == "max-age": - max_age_set = True - try: - v = int(v) - except ValueError: - _debug(" missing or invalid (non-numeric) value for " - "max-age attribute") - bad_cookie = True - break - # convert RFC 2965 Max-Age to seconds since epoch - # XXX Strictly you're supposed to follow RFC 2616 - # age-calculation rules. Remember that zero Max-Age is a - # is a request to discard (old and new) cookie, though. - k = "expires" - v = self._now + v - if (k in value_attrs) or (k in boolean_attrs): - if (v is None and - k not in ("port", "comment", "commenturl")): - _debug(" missing value for %s attribute" % k) - bad_cookie = True - break - standard[k] = v - else: - rest[k] = v - - if bad_cookie: - continue - - cookie_tuples.append((name, value, standard, rest)) - - return cookie_tuples - - def _cookie_from_cookie_tuple(self, tup, request): - # standard is dict of standard cookie-attributes, rest is dict of the - # rest of them - name, value, standard, rest = tup - - domain = standard.get("domain", Absent) - path = standard.get("path", Absent) - port = standard.get("port", Absent) - expires = standard.get("expires", Absent) - - # set the easy defaults - version = standard.get("version", None) - if version is not None: version = int(version) - secure = standard.get("secure", False) - # (discard is also set if expires is Absent) - discard = standard.get("discard", False) - comment = standard.get("comment", None) - comment_url = standard.get("commenturl", None) - - # set default path - if path is not Absent and path != "": - path_specified = True - path = escape_path(path) - else: - path_specified = False - path = request_path(request) - i = path.rfind("/") - if i != -1: - if version == 0: - # Netscape spec parts company from reality here - path = path[:i] - else: - path = path[:i+1] - if len(path) == 0: path = "/" - - # set default domain - domain_specified = domain is not Absent - # but first we have to remember whether it starts with a dot - domain_initial_dot = False - if domain_specified: - domain_initial_dot = bool(domain.startswith(".")) - if domain is Absent: - req_host, erhn = eff_request_host(request) - domain = erhn - elif not domain.startswith("."): - domain = "."+domain - - # set default port - port_specified = False - if port is not Absent: - if port is None: - # Port attr present, but has no value: default to request port. - # Cookie should then only be sent back on that port. - port = request_port(request) - else: - port_specified = True - port = re.sub(r"\s+", "", port) - else: - # No port attr present. Cookie can be sent back on any port. - port = None - - # set default expires and discard - if expires is Absent: - expires = None - discard = True - elif expires <= self._now: - # Expiry date in past is request to delete cookie. This can't be - # in DefaultCookiePolicy, because can't delete cookies there. - try: - self.clear(domain, path, name) - except KeyError: - pass - _debug("Expiring cookie, domain='%s', path='%s', name='%s'", - domain, path, name) - return None - - return Cookie(version, - name, value, - port, port_specified, - domain, domain_specified, domain_initial_dot, - path, path_specified, - secure, - expires, - discard, - comment, - comment_url, - rest) - - def _cookies_from_attrs_set(self, attrs_set, request): - cookie_tuples = self._normalized_cookie_tuples(attrs_set) - - cookies = [] - for tup in cookie_tuples: - cookie = self._cookie_from_cookie_tuple(tup, request) - if cookie: cookies.append(cookie) - return cookies - - def _process_rfc2109_cookies(self, cookies): - rfc2109_as_ns = getattr(self._policy, 'rfc2109_as_netscape', None) - if rfc2109_as_ns is None: - rfc2109_as_ns = not self._policy.rfc2965 - for cookie in cookies: - if cookie.version == 1: - cookie.rfc2109 = True - if rfc2109_as_ns: - # treat 2109 cookies as Netscape cookies rather than - # as RFC2965 cookies - cookie.version = 0 - - def make_cookies(self, response, request): - """Return sequence of Cookie objects extracted from response object.""" - # get cookie-attributes for RFC 2965 and Netscape protocols - headers = response.info() - rfc2965_hdrs = headers.getheaders("Set-Cookie2") - ns_hdrs = headers.getheaders("Set-Cookie") - - rfc2965 = self._policy.rfc2965 - netscape = self._policy.netscape - - if ((not rfc2965_hdrs and not ns_hdrs) or - (not ns_hdrs and not rfc2965) or - (not rfc2965_hdrs and not netscape) or - (not netscape and not rfc2965)): - return [] # no relevant cookie headers: quick exit - - try: - cookies = self._cookies_from_attrs_set( - split_header_words(rfc2965_hdrs), request) - except Exception: - _warn_unhandled_exception() - cookies = [] - - if ns_hdrs and netscape: - try: - # RFC 2109 and Netscape cookies - ns_cookies = self._cookies_from_attrs_set( - parse_ns_headers(ns_hdrs), request) - except Exception: - _warn_unhandled_exception() - ns_cookies = [] - self._process_rfc2109_cookies(ns_cookies) - - # Look for Netscape cookies (from Set-Cookie headers) that match - # corresponding RFC 2965 cookies (from Set-Cookie2 headers). - # For each match, keep the RFC 2965 cookie and ignore the Netscape - # cookie (RFC 2965 section 9.1). Actually, RFC 2109 cookies are - # bundled in with the Netscape cookies for this purpose, which is - # reasonable behaviour. - if rfc2965: - lookup = {} - for cookie in cookies: - lookup[(cookie.domain, cookie.path, cookie.name)] = None - - def no_matching_rfc2965(ns_cookie, lookup=lookup): - key = ns_cookie.domain, ns_cookie.path, ns_cookie.name - return key not in lookup - ns_cookies = filter(no_matching_rfc2965, ns_cookies) - - if ns_cookies: - cookies.extend(ns_cookies) - - return cookies - - def set_cookie_if_ok(self, cookie, request): - """Set a cookie if policy says it's OK to do so.""" - self._cookies_lock.acquire() - try: - self._policy._now = self._now = int(time.time()) - - if self._policy.set_ok(cookie, request): - self.set_cookie(cookie) - - - finally: - self._cookies_lock.release() - - def set_cookie(self, cookie): - """Set a cookie, without checking whether or not it should be set.""" - c = self._cookies - self._cookies_lock.acquire() - try: - if cookie.domain not in c: c[cookie.domain] = {} - c2 = c[cookie.domain] - if cookie.path not in c2: c2[cookie.path] = {} - c3 = c2[cookie.path] - c3[cookie.name] = cookie - finally: - self._cookies_lock.release() - - def extract_cookies(self, response, request): - """Extract cookies from response, where allowable given the request.""" - _debug("extract_cookies: %s", response.info()) - self._cookies_lock.acquire() - try: - self._policy._now = self._now = int(time.time()) - - for cookie in self.make_cookies(response, request): - if self._policy.set_ok(cookie, request): - _debug(" setting cookie: %s", cookie) - self.set_cookie(cookie) - finally: - self._cookies_lock.release() - - def clear(self, domain=None, path=None, name=None): - """Clear some cookies. - - Invoking this method without arguments will clear all cookies. If - given a single argument, only cookies belonging to that domain will be - removed. If given two arguments, cookies belonging to the specified - path within that domain are removed. If given three arguments, then - the cookie with the specified name, path and domain is removed. - - Raises KeyError if no matching cookie exists. - - """ - if name is not None: - if (domain is None) or (path is None): - raise ValueError( - "domain and path must be given to remove a cookie by name") - del self._cookies[domain][path][name] - elif path is not None: - if domain is None: - raise ValueError( - "domain must be given to remove cookies by path") - del self._cookies[domain][path] - elif domain is not None: - del self._cookies[domain] - else: - self._cookies = {} - - def clear_session_cookies(self): - """Discard all session cookies. - - Note that the .save() method won't save session cookies anyway, unless - you ask otherwise by passing a true ignore_discard argument. - - """ - self._cookies_lock.acquire() - try: - for cookie in self: - if cookie.discard: - self.clear(cookie.domain, cookie.path, cookie.name) - finally: - self._cookies_lock.release() - - def clear_expired_cookies(self): - """Discard all expired cookies. - - You probably don't need to call this method: expired cookies are never - sent back to the server (provided you're using DefaultCookiePolicy), - this method is called by CookieJar itself every so often, and the - .save() method won't save expired cookies anyway (unless you ask - otherwise by passing a true ignore_expires argument). - - """ - self._cookies_lock.acquire() - try: - now = time.time() - for cookie in self: - if cookie.is_expired(now): - self.clear(cookie.domain, cookie.path, cookie.name) - finally: - self._cookies_lock.release() - - def __iter__(self): - return deepvalues(self._cookies) - - def __len__(self): - """Return number of contained cookies.""" - i = 0 - for cookie in self: i = i + 1 - return i - - def __repr__(self): - r = [] - for cookie in self: r.append(repr(cookie)) - return "<%s[%s]>" % (self.__class__, ", ".join(r)) - - def __str__(self): - r = [] - for cookie in self: r.append(str(cookie)) - return "<%s[%s]>" % (self.__class__, ", ".join(r)) - - -# derives from IOError for backwards-compatibility with Python 2.4.0 -class LoadError(IOError): pass - -class FileCookieJar(CookieJar): - """CookieJar that can be loaded from and saved to a file.""" - - def __init__(self, filename=None, delayload=False, policy=None): - """ - Cookies are NOT loaded from the named file until either the .load() or - .revert() method is called. - - """ - CookieJar.__init__(self, policy) - if filename is not None: - try: - filename+"" - except: - raise ValueError("filename must be string-like") - self.filename = filename - self.delayload = bool(delayload) - - def save(self, filename=None, ignore_discard=False, ignore_expires=False): - """Save cookies to a file.""" - raise NotImplementedError() - - def load(self, filename=None, ignore_discard=False, ignore_expires=False): - """Load cookies from a file.""" - if filename is None: - if self.filename is not None: filename = self.filename - else: raise ValueError(MISSING_FILENAME_TEXT) - - f = open(filename) - try: - self._really_load(f, filename, ignore_discard, ignore_expires) - finally: - f.close() - - def revert(self, filename=None, - ignore_discard=False, ignore_expires=False): - """Clear all cookies and reload cookies from a saved file. - - Raises LoadError (or IOError) if reversion is not successful; the - object's state will not be altered if this happens. - - """ - if filename is None: - if self.filename is not None: filename = self.filename - else: raise ValueError(MISSING_FILENAME_TEXT) - - self._cookies_lock.acquire() - try: - - old_state = copy.deepcopy(self._cookies) - self._cookies = {} - try: - self.load(filename, ignore_discard, ignore_expires) - except (LoadError, IOError): - self._cookies = old_state - raise - - finally: - self._cookies_lock.release() - -from _LWPCookieJar import LWPCookieJar, lwp_cookie_str -from _MozillaCookieJar import MozillaCookieJar Modified: python/branches/py3k/Lib/distutils/command/upload.py ============================================================================== --- python/branches/py3k/Lib/distutils/command/upload.py (original) +++ python/branches/py3k/Lib/distutils/command/upload.py Mon May 26 18:32:26 2008 @@ -11,7 +11,7 @@ import socket import platform import configparser -import httplib +import http.client import base64 import urlparse @@ -151,9 +151,9 @@ urlparse.urlparse(self.repository) assert not params and not query and not fragments if schema == 'http': - http = httplib.HTTPConnection(netloc) + http = http.client.HTTPConnection(netloc) elif schema == 'https': - http = httplib.HTTPSConnection(netloc) + http = http.client.HTTPSConnection(netloc) else: raise AssertionError("unsupported schema "+schema) Added: python/branches/py3k/Lib/http/__init__.py ============================================================================== --- (empty file) +++ python/branches/py3k/Lib/http/__init__.py Mon May 26 18:32:26 2008 @@ -0,0 +1 @@ +# This directory is a Python package. Copied: python/branches/py3k/Lib/http/cookiejar.py (from r63693, /python/branches/py3k/Lib/cookielib.py) ============================================================================== --- /python/branches/py3k/Lib/cookielib.py (original) +++ python/branches/py3k/Lib/http/cookiejar.py Mon May 26 18:32:26 2008 @@ -33,7 +33,7 @@ import threading as _threading except ImportError: import dummy_threading as _threading -import httplib # only for the default HTTP port +import http.client # only for the default HTTP port from calendar import timegm debug = False # set to True to enable debugging via the logging module @@ -45,11 +45,11 @@ global logger if not logger: import logging - logger = logging.getLogger("cookielib") + logger = logging.getLogger("http.cookiejar") return logger.debug(*args) -DEFAULT_HTTP_PORT = str(httplib.HTTP_PORT) +DEFAULT_HTTP_PORT = str(http.client.HTTP_PORT) MISSING_FILENAME_TEXT = ("a filename was not supplied (nor was the CookieJar " "instance initialised with one)") @@ -61,7 +61,7 @@ f = io.StringIO() traceback.print_exc(None, f) msg = f.getvalue() - warnings.warn("cookielib bug!\n%s" % msg, stacklevel=2) + warnings.warn("http.cookiejar bug!\n%s" % msg, stacklevel=2) # Date/time conversion Copied: python/branches/py3k/Lib/http/cookies.py (from r63693, /python/branches/py3k/Lib/Cookie.py) ============================================================================== --- /python/branches/py3k/Lib/Cookie.py (original) +++ python/branches/py3k/Lib/http/cookies.py Mon May 26 18:32:26 2008 @@ -48,25 +48,25 @@ Importing is easy.. - >>> import Cookie + >>> from http import cookies Most of the time you start by creating a cookie. Cookies come in three flavors, each with slightly different encoding semantics, but more on that later. - >>> C = Cookie.SimpleCookie() - >>> C = Cookie.SerialCookie() - >>> C = Cookie.SmartCookie() + >>> C = cookies.SimpleCookie() + >>> C = cookies.SerialCookie() + >>> C = cookies.SmartCookie() -[Note: Long-time users of Cookie.py will remember using -Cookie.Cookie() to create an Cookie object. Although deprecated, it +[Note: Long-time users of cookies.py will remember using +cookies.Cookie() to create an Cookie object. Although deprecated, it is still supported by the code. See the Backward Compatibility notes for more information.] Once you've created your Cookie, you can add values just as if it were a dictionary. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C["fig"] = "newton" >>> C["sugar"] = "wafer" >>> C.output() @@ -77,7 +77,7 @@ default behavior. You can change the header and printed attributes by using the .output() function - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C["rocky"] = "road" >>> C["rocky"]["path"] = "/cookie" >>> print(C.output(header="Cookie:")) @@ -89,7 +89,7 @@ CGI script, you would use this method to extract the cookies from the HTTP_COOKIE environment variable. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C.load("chips=ahoy; vienna=finger") >>> C.output() 'Set-Cookie: chips=ahoy\r\nSet-Cookie: vienna=finger' @@ -98,7 +98,7 @@ within a string. Escaped quotation marks, nested semicolons, and other such trickeries do not confuse it. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";') >>> print(C) Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;" @@ -107,7 +107,7 @@ Cookie attributes. Here's an example which sets the Path attribute. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C["oreo"] = "doublestuff" >>> C["oreo"]["path"] = "/" >>> print(C) @@ -116,7 +116,7 @@ Each dictionary element has a 'value' attribute, which gives you back the value associated with the key. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C["twix"] = "none for you" >>> C["twix"].value 'none for you' @@ -135,7 +135,7 @@ Just to be sure, SimpleCookie invokes the str() builtin to convert the value to a string, when the values are set dictionary-style. - >>> C = Cookie.SimpleCookie() + >>> C = cookies.SimpleCookie() >>> C["number"] = 7 >>> C["string"] = "seven" >>> C["number"].value @@ -154,7 +154,7 @@ cookie has been returned. (SerialCookie can yield some strange-looking cookie values, however.) - >>> C = Cookie.SerialCookie() + >>> C = cookies.SerialCookie() >>> C["number"] = 7 >>> C["string"] = "seven" >>> C["number"].value @@ -178,7 +178,7 @@ the value. If it fails, then it fallsback to treating the value as a string. - >>> C = Cookie.SmartCookie() + >>> C = cookies.SmartCookie() >>> C["number"] = 7 >>> C["string"] = "seven" >>> C["number"].value @@ -193,10 +193,10 @@ ----------------------- In order to keep compatibilty with earlier versions of Cookie.py, -it is still possible to use Cookie.Cookie() to create a Cookie. In +it is still possible to use cookies.Cookie() to create a Cookie. In fact, this simply returns a SmartCookie. - >>> C = Cookie.Cookie() + >>> C = cookies.Cookie() >>> print(C.__class__.__name__) SmartCookie @@ -721,8 +721,8 @@ ########################################################### def _test(): - import doctest, Cookie - return doctest.testmod(Cookie) + import doctest, http.cookies + return doctest.testmod(http.cookies) if __name__ == "__main__": _test() Copied: python/branches/py3k/Lib/http/server.py (from r63693, /python/branches/py3k/Lib/BaseHTTPServer.py) ============================================================================== --- /python/branches/py3k/Lib/BaseHTTPServer.py (original) +++ python/branches/py3k/Lib/http/server.py Mon May 26 18:32:26 2008 @@ -1,14 +1,30 @@ -"""HTTP server base class. +"""HTTP server classes. -Note: the class in this module doesn't implement any HTTP request; see -SimpleHTTPServer for simple implementations of GET, HEAD and POST -(including CGI scripts). It does, however, optionally implement HTTP/1.1 -persistent connections, as of version 0.3. +Note: BaseHTTPRequestHandler doesn't implement any HTTP request; see +SimpleHTTPRequestHandler for simple implementations of GET, HEAD and POST, +and CGIHTTPRequestHandler for CGI scripts. -Contents: +It does, however, optionally implement HTTP/1.1 persistent connections, +as of version 0.3. -- BaseHTTPRequestHandler: HTTP request handler base class -- test: test function +Notes on CGIHTTPRequestHandler +------------------------------ + +This class implements GET and POST requests to cgi-bin scripts. + +If the os.fork() function is not present (e.g. on Windows), +os.popen2() is used as a fallback, with slightly altered semantics; if +that function is not present either (e.g. on Macintosh), only Python +scripts are supported, and they are executed by the current process. + +In all cases, the implementation is intentionally naive -- all +requests are executed synchronously. + +SECURITY WARNING: DON'T USE THIS CODE UNLESS YOU ARE INSIDE A FIREWALL +-- it may execute arbitrary Python code or external programs. + +Note that status code 200 is sent prior to execution of a CGI script, so +scripts cannot send other status codes such as 302 (redirect). XXX To do: @@ -66,15 +82,22 @@ # (Actually, the latter is only true if you know the server configuration # at the time the request was made!) -__version__ = "0.3" +__version__ = "0.6" __all__ = ["HTTPServer", "BaseHTTPRequestHandler"] import io +import os import sys +import cgi import time import socket # For gethostbyaddr() +import shutil +import urllib +import select import mimetools +import mimetypes +import posixpath import socketserver # Default error message template @@ -574,6 +597,521 @@ } +class SimpleHTTPRequestHandler(BaseHTTPRequestHandler): + + """Simple HTTP request handler with GET and HEAD commands. + + This serves files from the current directory and any of its + subdirectories. The MIME type for files is determined by + calling the .guess_type() method. + + The GET and HEAD requests are identical except that the HEAD + request omits the actual contents of the file. + + """ + + server_version = "SimpleHTTP/" + __version__ + + def do_GET(self): + """Serve a GET request.""" + f = self.send_head() + if f: + self.copyfile(f, self.wfile) + f.close() + + def do_HEAD(self): + """Serve a HEAD request.""" + f = self.send_head() + if f: + f.close() + + def send_head(self): + """Common code for GET and HEAD commands. + + This sends the response code and MIME headers. + + Return value is either a file object (which has to be copied + to the outputfile by the caller unless the command was HEAD, + and must be closed by the caller under all circumstances), or + None, in which case the caller has nothing further to do. + + """ + path = self.translate_path(self.path) + f = None + if os.path.isdir(path): + if not self.path.endswith('/'): + # redirect browser - doing basically what apache does + self.send_response(301) + self.send_header("Location", self.path + "/") + self.end_headers() + return None + for index in "index.html", "index.htm": + index = os.path.join(path, index) + if os.path.exists(index): + path = index + break + else: + return self.list_directory(path) + ctype = self.guess_type(path) + try: + f = open(path, 'rb') + except IOError: + self.send_error(404, "File not found") + return None + self.send_response(200) + self.send_header("Content-type", ctype) + fs = os.fstat(f.fileno()) + self.send_header("Content-Length", str(fs[6])) + self.send_header("Last-Modified", self.date_time_string(fs.st_mtime)) + self.end_headers() + return f + + def list_directory(self, path): + """Helper to produce a directory listing (absent index.html). + + Return value is either a file object, or None (indicating an + error). In either case, the headers are sent, making the + interface the same as for send_head(). + + """ + try: + list = os.listdir(path) + except os.error: + self.send_error(404, "No permission to list directory") + return None + list.sort(key=lambda a: a.lower()) + r = [] + displaypath = cgi.escape(urllib.unquote(self.path)) + r.append('') + r.append("\nDirectory listing for %s\n" % displaypath) + r.append("\n

Directory listing for %s

\n" % displaypath) + r.append("
\n
    \n") + for name in list: + fullname = os.path.join(path, name) + displayname = linkname = name + # Append / for directories or @ for symbolic links + if os.path.isdir(fullname): + displayname = name + "/" + linkname = name + "/" + if os.path.islink(fullname): + displayname = name + "@" + # Note: a link to a directory displays with @ and links with / + r.append('
  • %s\n' + % (urllib.quote(linkname), cgi.escape(displayname))) + r.append("
\n
\n\n\n") + enc = sys.getfilesystemencoding() + encoded = ''.join(r).encode(enc) + f = io.BytesIO() + f.write(encoded) + f.seek(0) + self.send_response(200) + self.send_header("Content-type", "text/html; charset=%s" % enc) + self.send_header("Content-Length", str(len(encoded))) + self.end_headers() + return f + + def translate_path(self, path): + """Translate a /-separated PATH to the local filename syntax. + + Components that mean special things to the local file system + (e.g. drive or directory names) are ignored. (XXX They should + probably be diagnosed.) + + """ + # abandon query parameters + path = path.split('?',1)[0] + path = path.split('#',1)[0] + path = posixpath.normpath(urllib.unquote(path)) + words = path.split('/') + words = filter(None, words) + path = os.getcwd() + for word in words: + drive, word = os.path.splitdrive(word) + head, word = os.path.split(word) + if word in (os.curdir, os.pardir): continue + path = os.path.join(path, word) + return path + + def copyfile(self, source, outputfile): + """Copy all data between two file objects. + + The SOURCE argument is a file object open for reading + (or anything with a read() method) and the DESTINATION + argument is a file object open for writing (or + anything with a write() method). + + The only reason for overriding this would be to change + the block size or perhaps to replace newlines by CRLF + -- note however that this the default server uses this + to copy binary data as well. + + """ + shutil.copyfileobj(source, outputfile) + + def guess_type(self, path): + """Guess the type of a file. + + Argument is a PATH (a filename). + + Return value is a string of the form type/subtype, + usable for a MIME Content-type header. + + The default implementation looks the file's extension + up in the table self.extensions_map, using application/octet-stream + as a default; however it would be permissible (if + slow) to look inside the data to make a better guess. + + """ + + base, ext = posixpath.splitext(path) + if ext in self.extensions_map: + return self.extensions_map[ext] + ext = ext.lower() + if ext in self.extensions_map: + return self.extensions_map[ext] + else: + return self.extensions_map[''] + + if not mimetypes.inited: + mimetypes.init() # try to read system mime.types + extensions_map = mimetypes.types_map.copy() + extensions_map.update({ + '': 'application/octet-stream', # Default + '.py': 'text/plain', + '.c': 'text/plain', + '.h': 'text/plain', + }) + + +# Utilities for CGIHTTPRequestHandler + +nobody = None + +def nobody_uid(): + """Internal routine to get nobody's uid""" + global nobody + if nobody: + return nobody + try: + import pwd + except ImportError: + return -1 + try: + nobody = pwd.getpwnam('nobody')[2] + except KeyError: + nobody = 1 + max(map(lambda x: x[2], pwd.getpwall())) + return nobody + + +def executable(path): + """Test for executable file.""" + try: + st = os.stat(path) + except os.error: + return False + return st.st_mode & 0o111 != 0 + + +class CGIHTTPRequestHandler(SimpleHTTPRequestHandler): + + """Complete HTTP server with GET, HEAD and POST commands. + + GET and HEAD also support running CGI scripts. + + The POST command is *only* implemented for CGI scripts. + + """ + + # Determine platform specifics + have_fork = hasattr(os, 'fork') + have_popen2 = hasattr(os, 'popen2') + have_popen3 = hasattr(os, 'popen3') + + # Make rfile unbuffered -- we need to read one line and then pass + # the rest to a subprocess, so we can't use buffered input. + rbufsize = 0 + + def do_POST(self): + """Serve a POST request. + + This is only implemented for CGI scripts. + + """ + + if self.is_cgi(): + self.run_cgi() + else: + self.send_error(501, "Can only POST to CGI scripts") + + def send_head(self): + """Version of send_head that support CGI scripts""" + if self.is_cgi(): + return self.run_cgi() + else: + return SimpleHTTPRequestHandler.send_head(self) + + def is_cgi(self): + """Test whether self.path corresponds to a CGI script. + + Return a tuple (dir, rest) if self.path requires running a + CGI script, None if not. Note that rest begins with a + slash if it is not empty. + + The default implementation tests whether the path + begins with one of the strings in the list + self.cgi_directories (and the next character is a '/' + or the end of the string). + + """ + + path = self.path + + for x in self.cgi_directories: + i = len(x) + if path[:i] == x and (not path[i:] or path[i] == '/'): + self.cgi_info = path[:i], path[i+1:] + return True + return False + + cgi_directories = ['/cgi-bin', '/htbin'] + + def is_executable(self, path): + """Test whether argument path is an executable file.""" + return executable(path) + + def is_python(self, path): + """Test whether argument path is a Python script.""" + head, tail = os.path.splitext(path) + return tail.lower() in (".py", ".pyw") + + def run_cgi(self): + """Execute a CGI script.""" + path = self.path + dir, rest = self.cgi_info + + i = path.find('/', len(dir) + 1) + while i >= 0: + nextdir = path[:i] + nextrest = path[i+1:] + + scriptdir = self.translate_path(nextdir) + if os.path.isdir(scriptdir): + dir, rest = nextdir, nextrest + i = path.find('/', len(dir) + 1) + else: + break + + # find an explicit query string, if present. + i = rest.rfind('?') + if i >= 0: + rest, query = rest[:i], rest[i+1:] + else: + query = '' + + # dissect the part after the directory name into a script name & + # a possible additional path, to be stored in PATH_INFO. + i = rest.find('/') + if i >= 0: + script, rest = rest[:i], rest[i:] + else: + script, rest = rest, '' + + scriptname = dir + '/' + script + scriptfile = self.translate_path(scriptname) + if not os.path.exists(scriptfile): + self.send_error(404, "No such CGI script (%r)" % scriptname) + return + if not os.path.isfile(scriptfile): + self.send_error(403, "CGI script is not a plain file (%r)" % + scriptname) + return + ispy = self.is_python(scriptname) + if not ispy: + if not (self.have_fork or self.have_popen2 or self.have_popen3): + self.send_error(403, "CGI script is not a Python script (%r)" % + scriptname) + return + if not self.is_executable(scriptfile): + self.send_error(403, "CGI script is not executable (%r)" % + scriptname) + return + + # Reference: http://hoohoo.ncsa.uiuc.edu/cgi/env.html + # XXX Much of the following could be prepared ahead of time! + env = {} + env['SERVER_SOFTWARE'] = self.version_string() + env['SERVER_NAME'] = self.server.server_name + env['GATEWAY_INTERFACE'] = 'CGI/1.1' + env['SERVER_PROTOCOL'] = self.protocol_version + env['SERVER_PORT'] = str(self.server.server_port) + env['REQUEST_METHOD'] = self.command + uqrest = urllib.unquote(rest) + env['PATH_INFO'] = uqrest + env['PATH_TRANSLATED'] = self.translate_path(uqrest) + env['SCRIPT_NAME'] = scriptname + if query: + env['QUERY_STRING'] = query + host = self.address_string() + if host != self.client_address[0]: + env['REMOTE_HOST'] = host + env['REMOTE_ADDR'] = self.client_address[0] + authorization = self.headers.getheader("authorization") + if authorization: + authorization = authorization.split() + if len(authorization) == 2: + import base64, binascii + env['AUTH_TYPE'] = authorization[0] + if authorization[0].lower() == "basic": + try: + authorization = authorization[1].encode('ascii') + authorization = base64.decodestring(authorization).\ + decode('ascii') + except (binascii.Error, UnicodeError): + pass + else: + authorization = authorization.split(':') + if len(authorization) == 2: + env['REMOTE_USER'] = authorization[0] + # XXX REMOTE_IDENT + if self.headers.typeheader is None: + env['CONTENT_TYPE'] = self.headers.type + else: + env['CONTENT_TYPE'] = self.headers.typeheader + length = self.headers.getheader('content-length') + if length: + env['CONTENT_LENGTH'] = length + referer = self.headers.getheader('referer') + if referer: + env['HTTP_REFERER'] = referer + accept = [] + for line in self.headers.getallmatchingheaders('accept'): + if line[:1] in "\t\n\r ": + accept.append(line.strip()) + else: + accept = accept + line[7:].split(',') + env['HTTP_ACCEPT'] = ','.join(accept) + ua = self.headers.getheader('user-agent') + if ua: + env['HTTP_USER_AGENT'] = ua + co = filter(None, self.headers.getheaders('cookie')) + if co: + env['HTTP_COOKIE'] = ', '.join(co) + # XXX Other HTTP_* headers + # Since we're setting the env in the parent, provide empty + # values to override previously set values + for k in ('QUERY_STRING', 'REMOTE_HOST', 'CONTENT_LENGTH', + 'HTTP_USER_AGENT', 'HTTP_COOKIE', 'HTTP_REFERER'): + env.setdefault(k, "") + os.environ.update(env) + + self.send_response(200, "Script output follows") + + decoded_query = query.replace('+', ' ') + + if self.have_fork: + # Unix -- fork as we should + args = [script] + if '=' not in decoded_query: + args.append(decoded_query) + nobody = nobody_uid() + self.wfile.flush() # Always flush before forking + pid = os.fork() + if pid != 0: + # Parent + pid, sts = os.waitpid(pid, 0) + # throw away additional data [see bug #427345] + while select.select([self.rfile], [], [], 0)[0]: + if not self.rfile.read(1): + break + if sts: + self.log_error("CGI script exit status %#x", sts) + return + # Child + try: + try: + os.setuid(nobody) + except os.error: + pass + os.dup2(self.rfile.fileno(), 0) + os.dup2(self.wfile.fileno(), 1) + os.execve(scriptfile, args, os.environ) + except: + self.server.handle_error(self.request, self.client_address) + os._exit(127) + + elif self.have_popen2 or self.have_popen3: + # Windows -- use popen2 or popen3 to create a subprocess + import shutil + if self.have_popen3: + popenx = os.popen3 + else: + popenx = os.popen2 + cmdline = scriptfile + if self.is_python(scriptfile): + interp = sys.executable + if interp.lower().endswith("w.exe"): + # On Windows, use python.exe, not pythonw.exe + interp = interp[:-5] + interp[-4:] + cmdline = "%s -u %s" % (interp, cmdline) + if '=' not in query and '"' not in query: + cmdline = '%s "%s"' % (cmdline, query) + self.log_message("command: %s", cmdline) + try: + nbytes = int(length) + except (TypeError, ValueError): + nbytes = 0 + files = popenx(cmdline, 'b') + fi = files[0] + fo = files[1] + if self.have_popen3: + fe = files[2] + if self.command.lower() == "post" and nbytes > 0: + data = self.rfile.read(nbytes) + fi.write(data) + # throw away additional data [see bug #427345] + while select.select([self.rfile._sock], [], [], 0)[0]: + if not self.rfile._sock.recv(1): + break + fi.close() + shutil.copyfileobj(fo, self.wfile) + if self.have_popen3: + errors = fe.read() + fe.close() + if errors: + self.log_error('%s', errors) + sts = fo.close() + if sts: + self.log_error("CGI script exit status %#x", sts) + else: + self.log_message("CGI script exited OK") + + else: + # Other O.S. -- execute script in this process + save_argv = sys.argv + save_stdin = sys.stdin + save_stdout = sys.stdout + save_stderr = sys.stderr + try: + save_cwd = os.getcwd() + try: + sys.argv = [scriptfile] + if '=' not in decoded_query: + sys.argv.append(decoded_query) + sys.stdout = self.wfile + sys.stdin = self.rfile + exec(open(scriptfile).read(), {"__name__": "__main__"}) + finally: + sys.argv = save_argv + sys.stdin = save_stdin + sys.stdout = save_stdout + sys.stderr = save_stderr + os.chdir(save_cwd) + except SystemExit as sts: + self.log_error("CGI script exit status %s", str(sts)) + else: + self.log_message("CGI script exited OK") + + def test(HandlerClass = BaseHTTPRequestHandler, ServerClass = HTTPServer, protocol="HTTP/1.0"): """Test the HTTP request handler class. @@ -598,4 +1136,6 @@ if __name__ == '__main__': - test() + test(HandlerClass=BaseHTTPRequestHandler) + test(HandlerClass=SimpleHTTPRequestHandler) + test(HandlerClass=CGIHTTPRequestHandler) Deleted: python/branches/py3k/Lib/httplib.py ============================================================================== --- python/branches/py3k/Lib/httplib.py Mon May 26 18:32:26 2008 +++ (empty file) @@ -1,1132 +0,0 @@ -"""HTTP/1.1 client library - - - - -HTTPConnection goes through a number of "states", which define when a client -may legally make another request or fetch the response for a particular -request. This diagram details these state transitions: - - (null) - | - | HTTPConnection() - v - Idle - | - | putrequest() - v - Request-started - | - | ( putheader() )* endheaders() - v - Request-sent - | - | response = getresponse() - v - Unread-response [Response-headers-read] - |\____________________ - | | - | response.read() | putrequest() - v v - Idle Req-started-unread-response - ______/| - / | - response.read() | | ( putheader() )* endheaders() - v v - Request-started Req-sent-unread-response - | - | response.read() - v - Request-sent - -This diagram presents the following rules: - -- a second request may not be started until {response-headers-read} - -- a response [object] cannot be retrieved until {request-sent} - -- there is no differentiation between an unread response body and a - partially read response body - -Note: this enforcement is applied by the HTTPConnection class. The - HTTPResponse class does not enforce this state machine, which - implies sophisticated clients may accelerate the request/response - pipeline. Caution should be taken, though: accelerating the states - beyond the above pattern may imply knowledge of the server's - connection-close behavior for certain requests. For example, it - is impossible to tell whether the server will close the connection - UNTIL the response headers have been read; this means that further - requests cannot be placed into the pipeline until it is known that - the server will NOT be closing the connection. - -Logical State __state __response -------------- ------- ---------- -Idle _CS_IDLE None -Request-started _CS_REQ_STARTED None -Request-sent _CS_REQ_SENT None -Unread-response _CS_IDLE -Req-started-unread-response _CS_REQ_STARTED -Req-sent-unread-response _CS_REQ_SENT -""" - -import io -import mimetools -import socket -from urlparse import urlsplit -import warnings - -__all__ = ["HTTPResponse", "HTTPConnection", - "HTTPException", "NotConnected", "UnknownProtocol", - "UnknownTransferEncoding", "UnimplementedFileMode", - "IncompleteRead", "InvalidURL", "ImproperConnectionState", - "CannotSendRequest", "CannotSendHeader", "ResponseNotReady", - "BadStatusLine", "error", "responses"] - -HTTP_PORT = 80 -HTTPS_PORT = 443 - -_UNKNOWN = 'UNKNOWN' - -# connection states -_CS_IDLE = 'Idle' -_CS_REQ_STARTED = 'Request-started' -_CS_REQ_SENT = 'Request-sent' - -# status codes -# informational -CONTINUE = 100 -SWITCHING_PROTOCOLS = 101 -PROCESSING = 102 - -# successful -OK = 200 -CREATED = 201 -ACCEPTED = 202 -NON_AUTHORITATIVE_INFORMATION = 203 -NO_CONTENT = 204 -RESET_CONTENT = 205 -PARTIAL_CONTENT = 206 -MULTI_STATUS = 207 -IM_USED = 226 - -# redirection -MULTIPLE_CHOICES = 300 -MOVED_PERMANENTLY = 301 -FOUND = 302 -SEE_OTHER = 303 -NOT_MODIFIED = 304 -USE_PROXY = 305 -TEMPORARY_REDIRECT = 307 - -# client error -BAD_REQUEST = 400 -UNAUTHORIZED = 401 -PAYMENT_REQUIRED = 402 -FORBIDDEN = 403 -NOT_FOUND = 404 -METHOD_NOT_ALLOWED = 405 -NOT_ACCEPTABLE = 406 -PROXY_AUTHENTICATION_REQUIRED = 407 -REQUEST_TIMEOUT = 408 -CONFLICT = 409 -GONE = 410 -LENGTH_REQUIRED = 411 -PRECONDITION_FAILED = 412 -REQUEST_ENTITY_TOO_LARGE = 413 -REQUEST_URI_TOO_LONG = 414 -UNSUPPORTED_MEDIA_TYPE = 415 -REQUESTED_RANGE_NOT_SATISFIABLE = 416 -EXPECTATION_FAILED = 417 -UNPROCESSABLE_ENTITY = 422 -LOCKED = 423 -FAILED_DEPENDENCY = 424 -UPGRADE_REQUIRED = 426 - -# server error -INTERNAL_SERVER_ERROR = 500 -NOT_IMPLEMENTED = 501 -BAD_GATEWAY = 502 -SERVICE_UNAVAILABLE = 503 -GATEWAY_TIMEOUT = 504 -HTTP_VERSION_NOT_SUPPORTED = 505 -INSUFFICIENT_STORAGE = 507 -NOT_EXTENDED = 510 - -# Mapping status codes to official W3C names -responses = { - 100: 'Continue', - 101: 'Switching Protocols', - - 200: 'OK', - 201: 'Created', - 202: 'Accepted', - 203: 'Non-Authoritative Information', - 204: 'No Content', - 205: 'Reset Content', - 206: 'Partial Content', - - 300: 'Multiple Choices', - 301: 'Moved Permanently', - 302: 'Found', - 303: 'See Other', - 304: 'Not Modified', - 305: 'Use Proxy', - 306: '(Unused)', - 307: 'Temporary Redirect', - - 400: 'Bad Request', - 401: 'Unauthorized', - 402: 'Payment Required', - 403: 'Forbidden', - 404: 'Not Found', - 405: 'Method Not Allowed', - 406: 'Not Acceptable', - 407: 'Proxy Authentication Required', - 408: 'Request Timeout', - 409: 'Conflict', - 410: 'Gone', - 411: 'Length Required', - 412: 'Precondition Failed', - 413: 'Request Entity Too Large', - 414: 'Request-URI Too Long', - 415: 'Unsupported Media Type', - 416: 'Requested Range Not Satisfiable', - 417: 'Expectation Failed', - - 500: 'Internal Server Error', - 501: 'Not Implemented', - 502: 'Bad Gateway', - 503: 'Service Unavailable', - 504: 'Gateway Timeout', - 505: 'HTTP Version Not Supported', -} - -# maximal amount of data to read at one time in _safe_read -MAXAMOUNT = 1048576 - -class HTTPMessage(mimetools.Message): - - def addheader(self, key, value): - """Add header for field key handling repeats.""" - prev = self.dict.get(key) - if prev is None: - self.dict[key] = value - else: - combined = ", ".join((prev, value)) - self.dict[key] = combined - - def addcontinue(self, key, more): - """Add more field data from a continuation line.""" - prev = self.dict[key] - self.dict[key] = prev + "\n " + more - - def readheaders(self): - """Read header lines. - - Read header lines up to the entirely blank line that terminates them. - The (normally blank) line that ends the headers is skipped, but not - included in the returned list. If a non-header line ends the headers, - (which is an error), an attempt is made to backspace over it; it is - never included in the returned list. - - The variable self.status is set to the empty string if all went well, - otherwise it is an error message. The variable self.headers is a - completely uninterpreted list of lines contained in the header (so - printing them will reproduce the header exactly as it appears in the - file). - - If multiple header fields with the same name occur, they are combined - according to the rules in RFC 2616 sec 4.2: - - Appending each subsequent field-value to the first, each separated - by a comma. The order in which header fields with the same field-name - are received is significant to the interpretation of the combined - field value. - """ - # XXX The implementation overrides the readheaders() method of - # rfc822.Message. The base class design isn't amenable to - # customized behavior here so the method here is a copy of the - # base class code with a few small changes. - - self.dict = {} - self.unixfrom = '' - self.headers = hlist = [] - self.status = '' - headerseen = "" - firstline = 1 - startofline = unread = tell = None - if hasattr(self.fp, 'unread'): - unread = self.fp.unread - elif self.seekable: - tell = self.fp.tell - while True: - if tell: - try: - startofline = tell() - except IOError: - startofline = tell = None - self.seekable = 0 - line = str(self.fp.readline(), "iso-8859-1") - if not line: - self.status = 'EOF in headers' - break - # Skip unix From name time lines - if firstline and line.startswith('From '): - self.unixfrom = self.unixfrom + line - continue - firstline = 0 - if headerseen and line[0] in ' \t': - # XXX Not sure if continuation lines are handled properly - # for http and/or for repeating headers - # It's a continuation line. - hlist.append(line) - self.addcontinue(headerseen, line.strip()) - continue - elif self.iscomment(line): - # It's a comment. Ignore it. - continue - elif self.islast(line): - # Note! No pushback here! The delimiter line gets eaten. - break - headerseen = self.isheader(line) - if headerseen: - # It's a legal header line, save it. - hlist.append(line) - self.addheader(headerseen, line[len(headerseen)+1:].strip()) - continue - else: - # It's not a header line; throw it back and stop here. - if not self.dict: - self.status = 'No headers' - else: - self.status = 'Non-header line where header expected' - # Try to undo the read. - if unread: - unread(line) - elif tell: - self.fp.seek(startofline) - else: - self.status = self.status + '; bad seek' - break - -class HTTPResponse: - - # strict: If true, raise BadStatusLine if the status line can't be - # parsed as a valid HTTP/1.0 or 1.1 status line. By default it is - # false because it prevents clients from talking to HTTP/0.9 - # servers. Note that a response with a sufficiently corrupted - # status line will look like an HTTP/0.9 response. - - # See RFC 2616 sec 19.6 and RFC 1945 sec 6 for details. - - # The bytes from the socket object are iso-8859-1 strings. - # See RFC 2616 sec 2.2 which notes an exception for MIME-encoded - # text following RFC 2047. The basic status line parsing only - # accepts iso-8859-1. - - def __init__(self, sock, debuglevel=0, strict=0, method=None): - # XXX If the response includes a content-length header, we - # need to make sure that the client doesn't read more than the - # specified number of bytes. If it does, it will block until - # the server times out and closes the connection. (The only - # applies to HTTP/1.1 connections.) Since some clients access - # self.fp directly rather than calling read(), this is a little - # tricky. - self.fp = sock.makefile("rb", 0) - self.debuglevel = debuglevel - self.strict = strict - self._method = method - - self.msg = None - - # from the Status-Line of the response - self.version = _UNKNOWN # HTTP-Version - self.status = _UNKNOWN # Status-Code - self.reason = _UNKNOWN # Reason-Phrase - - self.chunked = _UNKNOWN # is "chunked" being used? - self.chunk_left = _UNKNOWN # bytes left to read in current chunk - self.length = _UNKNOWN # number of bytes left in response - self.will_close = _UNKNOWN # conn will close at end of response - - def _read_status(self): - # Initialize with Simple-Response defaults. - line = str(self.fp.readline(), "iso-8859-1") - if self.debuglevel > 0: - print("reply:", repr(line)) - if not line: - # Presumably, the server closed the connection before - # sending a valid response. - raise BadStatusLine(line) - try: - [version, status, reason] = line.split(None, 2) - except ValueError: - try: - [version, status] = line.split(None, 1) - reason = "" - except ValueError: - # empty version will cause next test to fail and status - # will be treated as 0.9 response. - version = "" - if not version.startswith("HTTP/"): - if self.strict: - self.close() - raise BadStatusLine(line) - else: - # Assume it's a Simple-Response from an 0.9 server. - # We have to convert the first line back to raw bytes - # because self.fp.readline() needs to return bytes. - self.fp = LineAndFileWrapper(bytes(line, "ascii"), self.fp) - return "HTTP/0.9", 200, "" - - # The status code is a three-digit number - try: - status = int(status) - if status < 100 or status > 999: - raise BadStatusLine(line) - except ValueError: - raise BadStatusLine(line) - return version, status, reason - - def begin(self): - if self.msg is not None: - # we've already started reading the response - return - - # read until we get a non-100 response - while True: - version, status, reason = self._read_status() - if status != CONTINUE: - break - # skip the header from the 100 response - while True: - skip = self.fp.readline().strip() - if not skip: - break - if self.debuglevel > 0: - print("header:", skip) - - self.status = status - self.reason = reason.strip() - if version == "HTTP/1.0": - self.version = 10 - elif version.startswith("HTTP/1."): - self.version = 11 # use HTTP/1.1 code for HTTP/1.x where x>=1 - elif version == "HTTP/0.9": - self.version = 9 - else: - raise UnknownProtocol(version) - - if self.version == 9: - self.length = None - self.chunked = 0 - self.will_close = 1 - self.msg = HTTPMessage(io.BytesIO()) - return - - self.msg = HTTPMessage(self.fp, 0) - if self.debuglevel > 0: - for hdr in self.msg.headers: - print("header:", hdr, end=" ") - - # don't let the msg keep an fp - self.msg.fp = None - - # are we using the chunked-style of transfer encoding? - tr_enc = self.msg.getheader("transfer-encoding") - if tr_enc and tr_enc.lower() == "chunked": - self.chunked = 1 - self.chunk_left = None - else: - self.chunked = 0 - - # will the connection close at the end of the response? - self.will_close = self._check_close() - - # do we have a Content-Length? - # NOTE: RFC 2616, S4.4, #3 says we ignore this if tr_enc is "chunked" - self.length = None - length = self.msg.getheader("content-length") - if length and not self.chunked: - try: - self.length = int(length) - except ValueError: - self.length = None - else: - if self.length < 0: # ignore nonsensical negative lengths - self.length = None - else: - self.length = None - - # does the body have a fixed length? (of zero) - if (status == NO_CONTENT or status == NOT_MODIFIED or - 100 <= status < 200 or # 1xx codes - self._method == "HEAD"): - self.length = 0 - - # if the connection remains open, and we aren't using chunked, and - # a content-length was not provided, then assume that the connection - # WILL close. - if (not self.will_close and - not self.chunked and - self.length is None): - self.will_close = 1 - - def _check_close(self): - conn = self.msg.getheader("connection") - if self.version == 11: - # An HTTP/1.1 proxy is assumed to stay open unless - # explicitly closed. - conn = self.msg.getheader("connection") - if conn and "close" in conn.lower(): - return True - return False - - # Some HTTP/1.0 implementations have support for persistent - # connections, using rules different than HTTP/1.1. - - # For older HTTP, Keep-Alive indicates persistent connection. - if self.msg.getheader("keep-alive"): - return False - - # At least Akamai returns a "Connection: Keep-Alive" header, - # which was supposed to be sent by the client. - if conn and "keep-alive" in conn.lower(): - return False - - # Proxy-Connection is a netscape hack. - pconn = self.msg.getheader("proxy-connection") - if pconn and "keep-alive" in pconn.lower(): - return False - - # otherwise, assume it will close - return True - - def close(self): - if self.fp: - self.fp.close() - self.fp = None - - # These implementations are for the benefit of io.BufferedReader. - - # XXX This class should probably be revised to act more like - # the "raw stream" that BufferedReader expects. - - @property - def closed(self): - return self.isclosed() - - def flush(self): - self.fp.flush() - - # End of "raw stream" methods - - def isclosed(self): - # NOTE: it is possible that we will not ever call self.close(). This - # case occurs when will_close is TRUE, length is None, and we - # read up to the last byte, but NOT past it. - # - # IMPLIES: if will_close is FALSE, then self.close() will ALWAYS be - # called, meaning self.isclosed() is meaningful. - return self.fp is None - - # XXX It would be nice to have readline and __iter__ for this, too. - - def read(self, amt=None): - if self.fp is None: - return b"" - - if self.chunked: - return self._read_chunked(amt) - - if amt is None: - # unbounded read - if self.length is None: - s = self.fp.read() - else: - s = self._safe_read(self.length) - self.length = 0 - self.close() # we read everything - return s - - if self.length is not None: - if amt > self.length: - # clip the read to the "end of response" - amt = self.length - - # we do not use _safe_read() here because this may be a .will_close - # connection, and the user is reading more bytes than will be provided - # (for example, reading in 1k chunks) - s = self.fp.read(amt) - if self.length is not None: - self.length -= len(s) - if not self.length: - self.close() - return s - - def _read_chunked(self, amt): - assert self.chunked != _UNKNOWN - chunk_left = self.chunk_left - value = b"" - - # XXX This accumulates chunks by repeated string concatenation, - # which is not efficient as the number or size of chunks gets big. - while True: - if chunk_left is None: - line = self.fp.readline() - i = line.find(b";") - if i >= 0: - line = line[:i] # strip chunk-extensions - try: - chunk_left = int(line, 16) - except ValueError: - # close the connection as protocol synchronisation is - # probably lost - self.close() - raise IncompleteRead(value) - if chunk_left == 0: - break - if amt is None: - value += self._safe_read(chunk_left) - elif amt < chunk_left: - value += self._safe_read(amt) - self.chunk_left = chunk_left - amt - return value - elif amt == chunk_left: - value += self._safe_read(amt) - self._safe_read(2) # toss the CRLF at the end of the chunk - self.chunk_left = None - return value - else: - value += self._safe_read(chunk_left) - amt -= chunk_left - - # we read the whole chunk, get another - self._safe_read(2) # toss the CRLF at the end of the chunk - chunk_left = None - - # read and discard trailer up to the CRLF terminator - ### note: we shouldn't have any trailers! - while True: - line = self.fp.readline() - if not line: - # a vanishingly small number of sites EOF without - # sending the trailer - break - if line == b"\r\n": - break - - # we read everything; close the "file" - self.close() - - return value - - def _safe_read(self, amt): - """Read the number of bytes requested, compensating for partial reads. - - Normally, we have a blocking socket, but a read() can be interrupted - by a signal (resulting in a partial read). - - Note that we cannot distinguish between EOF and an interrupt when zero - bytes have been read. IncompleteRead() will be raised in this - situation. - - This function should be used when bytes "should" be present for - reading. If the bytes are truly not available (due to EOF), then the - IncompleteRead exception can be used to detect the problem. - """ - s = [] - while amt > 0: - chunk = self.fp.read(min(amt, MAXAMOUNT)) - if not chunk: - raise IncompleteRead(s) - s.append(chunk) - amt -= len(chunk) - return b"".join(s) - - def getheader(self, name, default=None): - if self.msg is None: - raise ResponseNotReady() - return self.msg.getheader(name, default) - - def getheaders(self): - """Return list of (header, value) tuples.""" - if self.msg is None: - raise ResponseNotReady() - return list(self.msg.items()) - - -class HTTPConnection: - - _http_vsn = 11 - _http_vsn_str = 'HTTP/1.1' - - response_class = HTTPResponse - default_port = HTTP_PORT - auto_open = 1 - debuglevel = 0 - strict = 0 - - def __init__(self, host, port=None, strict=None, timeout=None): - self.timeout = timeout - self.sock = None - self._buffer = [] - self.__response = None - self.__state = _CS_IDLE - self._method = None - - self._set_hostport(host, port) - if strict is not None: - self.strict = strict - - def _set_hostport(self, host, port): - if port is None: - i = host.rfind(':') - j = host.rfind(']') # ipv6 addresses have [...] - if i > j: - try: - port = int(host[i+1:]) - except ValueError: - raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) - host = host[:i] - else: - port = self.default_port - if host and host[0] == '[' and host[-1] == ']': - host = host[1:-1] - self.host = host - self.port = port - - def set_debuglevel(self, level): - self.debuglevel = level - - def connect(self): - """Connect to the host and port specified in __init__.""" - self.sock = socket.create_connection((self.host,self.port), - self.timeout) - - def close(self): - """Close the connection to the HTTP server.""" - if self.sock: - self.sock.close() # close it manually... there may be other refs - self.sock = None - if self.__response: - self.__response.close() - self.__response = None - self.__state = _CS_IDLE - - def send(self, str): - """Send `str' to the server.""" - if self.sock is None: - if self.auto_open: - self.connect() - else: - raise NotConnected() - - # send the data to the server. if we get a broken pipe, then close - # the socket. we want to reconnect when somebody tries to send again. - # - # NOTE: we DO propagate the error, though, because we cannot simply - # ignore the error... the caller will know if they can retry. - if self.debuglevel > 0: - print("send:", repr(str)) - try: - blocksize=8192 - if hasattr(str,'read') : - if self.debuglevel > 0: print("sendIng a read()able") - data=str.read(blocksize) - while data: - self.sock.sendall(data) - data=str.read(blocksize) - else: - self.sock.sendall(str) - except socket.error as v: - if v.args[0] == 32: # Broken pipe - self.close() - raise - - def _output(self, s): - """Add a line of output to the current request buffer. - - Assumes that the line does *not* end with \\r\\n. - """ - self._buffer.append(s) - - def _send_output(self): - """Send the currently buffered request and clear the buffer. - - Appends an extra \\r\\n to the buffer. - """ - self._buffer.extend((b"", b"")) - msg = b"\r\n".join(self._buffer) - del self._buffer[:] - self.send(msg) - - def putrequest(self, method, url, skip_host=0, skip_accept_encoding=0): - """Send a request to the server. - - `method' specifies an HTTP request method, e.g. 'GET'. - `url' specifies the object being requested, e.g. '/index.html'. - `skip_host' if True does not add automatically a 'Host:' header - `skip_accept_encoding' if True does not add automatically an - 'Accept-Encoding:' header - """ - - # if a prior response has been completed, then forget about it. - if self.__response and self.__response.isclosed(): - self.__response = None - - - # in certain cases, we cannot issue another request on this connection. - # this occurs when: - # 1) we are in the process of sending a request. (_CS_REQ_STARTED) - # 2) a response to a previous request has signalled that it is going - # to close the connection upon completion. - # 3) the headers for the previous response have not been read, thus - # we cannot determine whether point (2) is true. (_CS_REQ_SENT) - # - # if there is no prior response, then we can request at will. - # - # if point (2) is true, then we will have passed the socket to the - # response (effectively meaning, "there is no prior response"), and - # will open a new one when a new request is made. - # - # Note: if a prior response exists, then we *can* start a new request. - # We are not allowed to begin fetching the response to this new - # request, however, until that prior response is complete. - # - if self.__state == _CS_IDLE: - self.__state = _CS_REQ_STARTED - else: - raise CannotSendRequest() - - # Save the method we use, we need it later in the response phase - self._method = method - if not url: - url = '/' - request = '%s %s %s' % (method, url, self._http_vsn_str) - - # Non-ASCII characters should have been eliminated earlier - self._output(request.encode('ascii')) - - if self._http_vsn == 11: - # Issue some standard headers for better HTTP/1.1 compliance - - if not skip_host: - # this header is issued *only* for HTTP/1.1 - # connections. more specifically, this means it is - # only issued when the client uses the new - # HTTPConnection() class. backwards-compat clients - # will be using HTTP/1.0 and those clients may be - # issuing this header themselves. we should NOT issue - # it twice; some web servers (such as Apache) barf - # when they see two Host: headers - - # If we need a non-standard port,include it in the - # header. If the request is going through a proxy, - # but the host of the actual URL, not the host of the - # proxy. - - netloc = '' - if url.startswith('http'): - nil, netloc, nil, nil, nil = urlsplit(url) - - if netloc: - try: - netloc_enc = netloc.encode("ascii") - except UnicodeEncodeError: - netloc_enc = netloc.encode("idna") - self.putheader('Host', netloc_enc) - else: - try: - host_enc = self.host.encode("ascii") - except UnicodeEncodeError: - host_enc = self.host.encode("idna") - if self.port == HTTP_PORT: - self.putheader('Host', host_enc) - else: - host_enc = host_enc.decode("ascii") - self.putheader('Host', "%s:%s" % (host_enc, self.port)) - - # note: we are assuming that clients will not attempt to set these - # headers since *this* library must deal with the - # consequences. this also means that when the supporting - # libraries are updated to recognize other forms, then this - # code should be changed (removed or updated). - - # we only want a Content-Encoding of "identity" since we don't - # support encodings such as x-gzip or x-deflate. - if not skip_accept_encoding: - self.putheader('Accept-Encoding', 'identity') - - # we can accept "chunked" Transfer-Encodings, but no others - # NOTE: no TE header implies *only* "chunked" - #self.putheader('TE', 'chunked') - - # if TE is supplied in the header, then it must appear in a - # Connection header. - #self.putheader('Connection', 'TE') - - else: - # For HTTP/1.0, the server will assume "not chunked" - pass - - def putheader(self, header, value): - """Send a request header line to the server. - - For example: h.putheader('Accept', 'text/html') - """ - if self.__state != _CS_REQ_STARTED: - raise CannotSendHeader() - - if hasattr(header, 'encode'): - header = header.encode('ascii') - if hasattr(value, 'encode'): - value = value.encode('ascii') - header = header + b': ' + value - self._output(header) - - def endheaders(self): - """Indicate that the last header line has been sent to the server.""" - - if self.__state == _CS_REQ_STARTED: - self.__state = _CS_REQ_SENT - else: - raise CannotSendHeader() - - self._send_output() - - def request(self, method, url, body=None, headers={}): - """Send a complete request to the server.""" - try: - self._send_request(method, url, body, headers) - except socket.error as v: - # trap 'Broken pipe' if we're allowed to automatically reconnect - if v.args[0] != 32 or not self.auto_open: - raise - # try one more time - self._send_request(method, url, body, headers) - - def _send_request(self, method, url, body, headers): - # honour explicitly requested Host: and Accept-Encoding headers - header_names = dict.fromkeys([k.lower() for k in headers]) - skips = {} - if 'host' in header_names: - skips['skip_host'] = 1 - if 'accept-encoding' in header_names: - skips['skip_accept_encoding'] = 1 - - self.putrequest(method, url, **skips) - - if body and ('content-length' not in header_names): - thelen = None - try: - thelen = str(len(body)) - except TypeError as te: - # If this is a file-like object, try to - # fstat its file descriptor - import os - try: - thelen = str(os.fstat(body.fileno()).st_size) - except (AttributeError, OSError): - # Don't send a length if this failed - if self.debuglevel > 0: print("Cannot stat!!") - - if thelen is not None: - self.putheader('Content-Length',thelen) - for hdr, value in headers.items(): - self.putheader(hdr, value) - self.endheaders() - - if body: - if isinstance(body, str): body = body.encode('ascii') - self.send(body) - - def getresponse(self): - """Get the response from the server.""" - - # if a prior response has been completed, then forget about it. - if self.__response and self.__response.isclosed(): - self.__response = None - - # - # if a prior response exists, then it must be completed (otherwise, we - # cannot read this response's header to determine the connection-close - # behavior) - # - # note: if a prior response existed, but was connection-close, then the - # socket and response were made independent of this HTTPConnection - # object since a new request requires that we open a whole new - # connection - # - # this means the prior response had one of two states: - # 1) will_close: this connection was reset and the prior socket and - # response operate independently - # 2) persistent: the response was retained and we await its - # isclosed() status to become true. - # - if self.__state != _CS_REQ_SENT or self.__response: - raise ResponseNotReady() - - if self.debuglevel > 0: - response = self.response_class(self.sock, self.debuglevel, - strict=self.strict, - method=self._method) - else: - response = self.response_class(self.sock, strict=self.strict, - method=self._method) - - response.begin() - assert response.will_close != _UNKNOWN - self.__state = _CS_IDLE - - if response.will_close: - # this effectively passes the connection to the response - self.close() - else: - # remember this, so we can tell when it is complete - self.__response = response - - return response - -try: - import ssl -except ImportError: - pass -else: - class HTTPSConnection(HTTPConnection): - "This class allows communication via SSL." - - default_port = HTTPS_PORT - - def __init__(self, host, port=None, key_file=None, cert_file=None, - strict=None, timeout=None): - HTTPConnection.__init__(self, host, port, strict, timeout) - self.key_file = key_file - self.cert_file = cert_file - - def connect(self): - "Connect to a host on a given (SSL) port." - - sock = socket.create_connection((self.host, self.port), self.timeout) - self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file) - - - def FakeSocket (sock, sslobj): - warnings.warn("FakeSocket is deprecated, and won't be in 3.x. " + - "Use the result of ssl.wrap_socket() directly instead.", - DeprecationWarning, stacklevel=2) - return sslobj - - __all__.append("HTTPSConnection") - -class HTTPException(Exception): - # Subclasses that define an __init__ must call Exception.__init__ - # or define self.args. Otherwise, str() will fail. - pass - -class NotConnected(HTTPException): - pass - -class InvalidURL(HTTPException): - pass - -class UnknownProtocol(HTTPException): - def __init__(self, version): - self.args = version, - self.version = version - -class UnknownTransferEncoding(HTTPException): - pass - -class UnimplementedFileMode(HTTPException): - pass - -class IncompleteRead(HTTPException): - def __init__(self, partial): - self.args = partial, - self.partial = partial - -class ImproperConnectionState(HTTPException): - pass - -class CannotSendRequest(ImproperConnectionState): - pass - -class CannotSendHeader(ImproperConnectionState): - pass - -class ResponseNotReady(ImproperConnectionState): - pass - -class BadStatusLine(HTTPException): - def __init__(self, line): - self.args = line, - self.line = line - -# for backwards compatibility -error = HTTPException - -class LineAndFileWrapper: - """A limited file-like object for HTTP/0.9 responses.""" - - # The status-line parsing code calls readline(), which normally - # get the HTTP status line. For a 0.9 response, however, this is - # actually the first line of the body! Clients need to get a - # readable file object that contains that line. - - def __init__(self, line, file): - self._line = line - self._file = file - self._line_consumed = 0 - self._line_offset = 0 - self._line_left = len(line) - - def __getattr__(self, attr): - return getattr(self._file, attr) - - def _done(self): - # called when the last byte is read from the line. After the - # call, all read methods are delegated to the underlying file - # object. - self._line_consumed = 1 - self.read = self._file.read - self.readline = self._file.readline - self.readlines = self._file.readlines - - def read(self, amt=None): - if self._line_consumed: - return self._file.read(amt) - assert self._line_left - if amt is None or amt > self._line_left: - s = self._line[self._line_offset:] - self._done() - if amt is None: - return s + self._file.read() - else: - return s + self._file.read(amt - len(s)) - else: - assert amt <= self._line_left - i = self._line_offset - j = i + amt - s = self._line[i:j] - self._line_offset = j - self._line_left -= amt - if self._line_left == 0: - self._done() - return s - - def readline(self): - if self._line_consumed: - return self._file.readline() - assert self._line_left - s = self._line[self._line_offset:] - self._done() - return s - - def readlines(self, size=None): - if self._line_consumed: - return self._file.readlines(size) - assert self._line_left - L = [self._line[self._line_offset:]] - self._done() - if size is None: - return L + self._file.readlines() - else: - return L + self._file.readlines(size) Modified: python/branches/py3k/Lib/logging/handlers.py ============================================================================== --- python/branches/py3k/Lib/logging/handlers.py (original) +++ python/branches/py3k/Lib/logging/handlers.py Mon May 26 18:32:26 2008 @@ -1002,9 +1002,9 @@ Send the record to the Web server as an URL-encoded dictionary """ try: - import httplib, urllib + import http.client, urllib host = self.host - h = httplib.HTTP(host) + h = http.client.HTTP(host) url = self.url data = urllib.urlencode(self.mapLogRecord(record)) if self.method == "GET": Modified: python/branches/py3k/Lib/pydoc.py ============================================================================== --- python/branches/py3k/Lib/pydoc.py (original) +++ python/branches/py3k/Lib/pydoc.py Mon May 26 18:32:26 2008 @@ -1921,7 +1921,7 @@ # --------------------------------------------------- web browser interface def serve(port, callback=None, completer=None): - import BaseHTTPServer, mimetools, select + import http.server, mimetools, select # Patch up mimetools.Message so it doesn't break if rfc822 is reloaded. class Message(mimetools.Message): @@ -1933,7 +1933,7 @@ self.parsetype() self.parseplist() - class DocHandler(BaseHTTPServer.BaseHTTPRequestHandler): + class DocHandler(http.server.BaseHTTPRequestHandler): def send_document(self, title, contents): try: self.send_response(200) @@ -1978,7 +1978,7 @@ def log_message(self, *args): pass - class DocServer(BaseHTTPServer.HTTPServer): + class DocServer(http.server.HTTPServer): def __init__(self, port, callback): host = (sys.platform == 'mac') and '127.0.0.1' or 'localhost' self.address = ('', port) @@ -1997,7 +1997,7 @@ self.base.server_activate(self) if self.callback: self.callback(self) - DocServer.base = BaseHTTPServer.HTTPServer + DocServer.base = http.server.HTTPServer DocServer.handler = DocHandler DocHandler.MessageClass = Message try: Modified: python/branches/py3k/Lib/test/test_SimpleHTTPServer.py ============================================================================== --- python/branches/py3k/Lib/test/test_SimpleHTTPServer.py (original) +++ python/branches/py3k/Lib/test/test_SimpleHTTPServer.py Mon May 26 18:32:26 2008 @@ -4,11 +4,11 @@ """ import os, unittest -from SimpleHTTPServer import SimpleHTTPRequestHandler +from http.server import SimpleHTTPRequestHandler from test import support -class SocketlessRequestHandler (SimpleHTTPRequestHandler): +class SocketlessRequestHandler(SimpleHTTPRequestHandler): def __init__(self): pass Modified: python/branches/py3k/Lib/test/test___all__.py ============================================================================== --- python/branches/py3k/Lib/test/test___all__.py (original) +++ python/branches/py3k/Lib/test/test___all__.py Mon May 26 18:32:26 2008 @@ -33,12 +33,10 @@ # than an AttributeError somewhere deep in CGIHTTPServer. import _socket - self.check_all("BaseHTTPServer") - self.check_all("CGIHTTPServer") + self.check_all("http.server") self.check_all("configparser") - self.check_all("Cookie") - self.check_all("Queue") - self.check_all("SimpleHTTPServer") + self.check_all("http.cookies") + self.check_all("queue") self.check_all("socketserver") self.check_all("aifc") self.check_all("base64") @@ -77,7 +75,7 @@ self.check_all("gzip") self.check_all("heapq") self.check_all("htmllib") - self.check_all("httplib") + self.check_all("http.client") self.check_all("ihooks") self.check_all("imaplib") self.check_all("imghdr") Deleted: python/branches/py3k/Lib/test/test_cookie.py ============================================================================== --- python/branches/py3k/Lib/test/test_cookie.py Mon May 26 18:32:26 2008 +++ (empty file) @@ -1,82 +0,0 @@ -# Simple test suite for Cookie.py - -from test.support import run_unittest, run_doctest -import unittest -import Cookie - -import warnings -warnings.filterwarnings("ignore", - ".* class is insecure.*", - DeprecationWarning) - -class CookieTests(unittest.TestCase): - # Currently this only tests SimpleCookie - def test_basic(self): - cases = [ - { 'data': 'chips=ahoy; vienna=finger', - 'dict': {'chips':'ahoy', 'vienna':'finger'}, - 'repr': "", - 'output': 'Set-Cookie: chips=ahoy\nSet-Cookie: vienna=finger', - }, - - { 'data': 'keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"', - 'dict': {'keebler' : 'E=mc2; L="Loves"; fudge=\012;'}, - 'repr': '''''', - 'output': 'Set-Cookie: keebler="E=mc2; L=\\"Loves\\"; fudge=\\012;"', - }, - - # Check illegal cookies that have an '=' char in an unquoted value - { 'data': 'keebler=E=mc2', - 'dict': {'keebler' : 'E=mc2'}, - 'repr': "", - 'output': 'Set-Cookie: keebler=E=mc2', - } - ] - - for case in cases: - C = Cookie.SimpleCookie() - C.load(case['data']) - self.assertEqual(repr(C), case['repr']) - self.assertEqual(C.output(sep='\n'), case['output']) - for k, v in sorted(case['dict'].items()): - self.assertEqual(C[k].value, v) - - def test_load(self): - C = Cookie.SimpleCookie() - C.load('Customer="WILE_E_COYOTE"; Version=1; Path=/acme') - - self.assertEqual(C['Customer'].value, 'WILE_E_COYOTE') - self.assertEqual(C['Customer']['version'], '1') - self.assertEqual(C['Customer']['path'], '/acme') - - self.assertEqual(C.output(['path']), - 'Set-Cookie: Customer="WILE_E_COYOTE"; Path=/acme') - self.assertEqual(C.js_output(), """ - - """) - self.assertEqual(C.js_output(['path']), """ - - """) - - def test_quoted_meta(self): - # Try cookie with quoted meta-data - C = Cookie.SimpleCookie() - C.load('Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"') - self.assertEqual(C['Customer'].value, 'WILE_E_COYOTE') - self.assertEqual(C['Customer']['version'], '1') - self.assertEqual(C['Customer']['path'], '/acme') - -def test_main(): - run_unittest(CookieTests) - run_doctest(Cookie) - -if __name__ == '__main__': - test_main() Deleted: python/branches/py3k/Lib/test/test_cookielib.py ============================================================================== --- python/branches/py3k/Lib/test/test_cookielib.py Mon May 26 18:32:26 2008 +++ (empty file) @@ -1,1734 +0,0 @@ -"""Tests for cookielib.py.""" - -import re, os, time -from unittest import TestCase - -from test import support - -class DateTimeTests(TestCase): - - def test_time2isoz(self): - from cookielib import time2isoz - - base = 1019227000 - day = 24*3600 - self.assertEquals(time2isoz(base), "2002-04-19 14:36:40Z") - self.assertEquals(time2isoz(base+day), "2002-04-20 14:36:40Z") - self.assertEquals(time2isoz(base+2*day), "2002-04-21 14:36:40Z") - self.assertEquals(time2isoz(base+3*day), "2002-04-22 14:36:40Z") - - az = time2isoz() - bz = time2isoz(500000) - for text in (az, bz): - self.assert_(re.search(r"^\d{4}-\d\d-\d\d \d\d:\d\d:\d\dZ$", text), - "bad time2isoz format: %s %s" % (az, bz)) - - def test_http2time(self): - from cookielib import http2time - - def parse_date(text): - return time.gmtime(http2time(text))[:6] - - self.assertEquals(parse_date("01 Jan 2001"), (2001, 1, 1, 0, 0, 0.0)) - - # this test will break around year 2070 - self.assertEquals(parse_date("03-Feb-20"), (2020, 2, 3, 0, 0, 0.0)) - - # this test will break around year 2048 - self.assertEquals(parse_date("03-Feb-98"), (1998, 2, 3, 0, 0, 0.0)) - - def test_http2time_formats(self): - from cookielib import http2time, time2isoz - - # test http2time for supported dates. Test cases with 2 digit year - # will probably break in year 2044. - tests = [ - 'Thu, 03 Feb 1994 00:00:00 GMT', # proposed new HTTP format - 'Thursday, 03-Feb-94 00:00:00 GMT', # old rfc850 HTTP format - 'Thursday, 03-Feb-1994 00:00:00 GMT', # broken rfc850 HTTP format - - '03 Feb 1994 00:00:00 GMT', # HTTP format (no weekday) - '03-Feb-94 00:00:00 GMT', # old rfc850 (no weekday) - '03-Feb-1994 00:00:00 GMT', # broken rfc850 (no weekday) - '03-Feb-1994 00:00 GMT', # broken rfc850 (no weekday, no seconds) - '03-Feb-1994 00:00', # broken rfc850 (no weekday, no seconds, no tz) - - '03-Feb-94', # old rfc850 HTTP format (no weekday, no time) - '03-Feb-1994', # broken rfc850 HTTP format (no weekday, no time) - '03 Feb 1994', # proposed new HTTP format (no weekday, no time) - - # A few tests with extra space at various places - ' 03 Feb 1994 0:00 ', - ' 03-Feb-1994 ', - ] - - test_t = 760233600 # assume broken POSIX counting of seconds - result = time2isoz(test_t) - expected = "1994-02-03 00:00:00Z" - self.assertEquals(result, expected, - "%s => '%s' (%s)" % (test_t, result, expected)) - - for s in tests: - t = http2time(s) - t2 = http2time(s.lower()) - t3 = http2time(s.upper()) - - self.assert_(t == t2 == t3 == test_t, - "'%s' => %s, %s, %s (%s)" % (s, t, t2, t3, test_t)) - - def test_http2time_garbage(self): - from cookielib import http2time - - for test in [ - '', - 'Garbage', - 'Mandag 16. September 1996', - '01-00-1980', - '01-13-1980', - '00-01-1980', - '32-01-1980', - '01-01-1980 25:00:00', - '01-01-1980 00:61:00', - '01-01-1980 00:00:62', - ]: - self.assert_(http2time(test) is None, - "http2time(%s) is not None\n" - "http2time(test) %s" % (test, http2time(test)) - ) - - -class HeaderTests(TestCase): - def test_parse_ns_headers(self): - from cookielib import parse_ns_headers - - # quotes should be stripped - expected = [[('foo', 'bar'), ('expires', 2209069412), ('version', '0')]] - for hdr in [ - 'foo=bar; expires=01 Jan 2040 22:23:32 GMT', - 'foo=bar; expires="01 Jan 2040 22:23:32 GMT"', - ]: - self.assertEquals(parse_ns_headers([hdr]), expected) - - def test_parse_ns_headers_special_names(self): - # names such as 'expires' are not special in first name=value pair - # of Set-Cookie: header - from cookielib import parse_ns_headers - - # Cookie with name 'expires' - hdr = 'expires=01 Jan 2040 22:23:32 GMT' - expected = [[("expires", "01 Jan 2040 22:23:32 GMT"), ("version", "0")]] - self.assertEquals(parse_ns_headers([hdr]), expected) - - def test_join_header_words(self): - from cookielib import join_header_words - - joined = join_header_words([[("foo", None), ("bar", "baz")]]) - self.assertEquals(joined, "foo; bar=baz") - - self.assertEquals(join_header_words([[]]), "") - - def test_split_header_words(self): - from cookielib import split_header_words - - tests = [ - ("foo", [[("foo", None)]]), - ("foo=bar", [[("foo", "bar")]]), - (" foo ", [[("foo", None)]]), - (" foo= ", [[("foo", "")]]), - (" foo=", [[("foo", "")]]), - (" foo= ; ", [[("foo", "")]]), - (" foo= ; bar= baz ", [[("foo", ""), ("bar", "baz")]]), - ("foo=bar bar=baz", [[("foo", "bar"), ("bar", "baz")]]), - # doesn't really matter if this next fails, but it works ATM - ("foo= bar=baz", [[("foo", "bar=baz")]]), - ("foo=bar;bar=baz", [[("foo", "bar"), ("bar", "baz")]]), - ('foo bar baz', [[("foo", None), ("bar", None), ("baz", None)]]), - ("a, b, c", [[("a", None)], [("b", None)], [("c", None)]]), - (r'foo; bar=baz, spam=, foo="\,\;\"", bar= ', - [[("foo", None), ("bar", "baz")], - [("spam", "")], [("foo", ',;"')], [("bar", "")]]), - ] - - for arg, expect in tests: - try: - result = split_header_words([arg]) - except: - import traceback, io - f = io.StringIO() - traceback.print_exc(None, f) - result = "(error -- traceback follows)\n\n%s" % f.getvalue() - self.assertEquals(result, expect, """ -When parsing: '%s' -Expected: '%s' -Got: '%s' -""" % (arg, expect, result)) - - def test_roundtrip(self): - from cookielib import split_header_words, join_header_words - - tests = [ - ("foo", "foo"), - ("foo=bar", "foo=bar"), - (" foo ", "foo"), - ("foo=", 'foo=""'), - ("foo=bar bar=baz", "foo=bar; bar=baz"), - ("foo=bar;bar=baz", "foo=bar; bar=baz"), - ('foo bar baz', "foo; bar; baz"), - (r'foo="\"" bar="\\"', r'foo="\""; bar="\\"'), - ('foo,,,bar', 'foo, bar'), - ('foo=bar,bar=baz', 'foo=bar, bar=baz'), - - ('text/html; charset=iso-8859-1', - 'text/html; charset="iso-8859-1"'), - - ('foo="bar"; port="80,81"; discard, bar=baz', - 'foo=bar; port="80,81"; discard, bar=baz'), - - (r'Basic realm="\"foo\\\\bar\""', - r'Basic; realm="\"foo\\\\bar\""') - ] - - for arg, expect in tests: - input = split_header_words([arg]) - res = join_header_words(input) - self.assertEquals(res, expect, """ -When parsing: '%s' -Expected: '%s' -Got: '%s' -Input was: '%s' -""" % (arg, expect, res, input)) - - -class FakeResponse: - def __init__(self, headers=[], url=None): - """ - headers: list of RFC822-style 'Key: value' strings - """ - import mimetools, io - f = io.StringIO("\n".join(headers)) - self._headers = mimetools.Message(f) - self._url = url - def info(self): return self._headers - -def interact_2965(cookiejar, url, *set_cookie_hdrs): - return _interact(cookiejar, url, set_cookie_hdrs, "Set-Cookie2") - -def interact_netscape(cookiejar, url, *set_cookie_hdrs): - return _interact(cookiejar, url, set_cookie_hdrs, "Set-Cookie") - -def _interact(cookiejar, url, set_cookie_hdrs, hdr_name): - """Perform a single request / response cycle, returning Cookie: header.""" - from urllib2 import Request - req = Request(url) - cookiejar.add_cookie_header(req) - cookie_hdr = req.get_header("Cookie", "") - headers = [] - for hdr in set_cookie_hdrs: - headers.append("%s: %s" % (hdr_name, hdr)) - res = FakeResponse(headers, url) - cookiejar.extract_cookies(res, req) - return cookie_hdr - - -class FileCookieJarTests(TestCase): - def test_lwp_valueless_cookie(self): - # cookies with no value should be saved and loaded consistently - from cookielib import LWPCookieJar - filename = support.TESTFN - c = LWPCookieJar() - interact_netscape(c, "http://www.acme.com/", 'boo') - self.assertEqual(c._cookies["www.acme.com"]["/"]["boo"].value, None) - try: - c.save(filename, ignore_discard=True) - c = LWPCookieJar() - c.load(filename, ignore_discard=True) - finally: - try: os.unlink(filename) - except OSError: pass - self.assertEqual(c._cookies["www.acme.com"]["/"]["boo"].value, None) - - def test_bad_magic(self): - from cookielib import LWPCookieJar, MozillaCookieJar, LoadError - # IOErrors (eg. file doesn't exist) are allowed to propagate - filename = support.TESTFN - for cookiejar_class in LWPCookieJar, MozillaCookieJar: - c = cookiejar_class() - try: - c.load(filename="for this test to work, a file with this " - "filename should not exist") - except IOError as exc: - # exactly IOError, not LoadError - self.assertEqual(exc.__class__, IOError) - else: - self.fail("expected IOError for invalid filename") - # Invalid contents of cookies file (eg. bad magic string) - # causes a LoadError. - try: - f = open(filename, "w") - f.write("oops\n") - for cookiejar_class in LWPCookieJar, MozillaCookieJar: - c = cookiejar_class() - self.assertRaises(LoadError, c.load, filename) - finally: - try: os.unlink(filename) - except OSError: pass - -class CookieTests(TestCase): - # XXX - # Get rid of string comparisons where not actually testing str / repr. - # .clear() etc. - # IP addresses like 50 (single number, no dot) and domain-matching - # functions (and is_HDN)? See draft RFC 2965 errata. - # Strictness switches - # is_third_party() - # unverifiability / third-party blocking - # Netscape cookies work the same as RFC 2965 with regard to port. - # Set-Cookie with negative max age. - # If turn RFC 2965 handling off, Set-Cookie2 cookies should not clobber - # Set-Cookie cookies. - # Cookie2 should be sent if *any* cookies are not V1 (ie. V0 OR V2 etc.). - # Cookies (V1 and V0) with no expiry date should be set to be discarded. - # RFC 2965 Quoting: - # Should accept unquoted cookie-attribute values? check errata draft. - # Which are required on the way in and out? - # Should always return quoted cookie-attribute values? - # Proper testing of when RFC 2965 clobbers Netscape (waiting for errata). - # Path-match on return (same for V0 and V1). - # RFC 2965 acceptance and returning rules - # Set-Cookie2 without version attribute is rejected. - - # Netscape peculiarities list from Ronald Tschalar. - # The first two still need tests, the rest are covered. -## - Quoting: only quotes around the expires value are recognized as such -## (and yes, some folks quote the expires value); quotes around any other -## value are treated as part of the value. -## - White space: white space around names and values is ignored -## - Default path: if no path parameter is given, the path defaults to the -## path in the request-uri up to, but not including, the last '/'. Note -## that this is entirely different from what the spec says. -## - Commas and other delimiters: Netscape just parses until the next ';'. -## This means it will allow commas etc inside values (and yes, both -## commas and equals are commonly appear in the cookie value). This also -## means that if you fold multiple Set-Cookie header fields into one, -## comma-separated list, it'll be a headache to parse (at least my head -## starts hurting everytime I think of that code). -## - Expires: You'll get all sorts of date formats in the expires, -## including emtpy expires attributes ("expires="). Be as flexible as you -## can, and certainly don't expect the weekday to be there; if you can't -## parse it, just ignore it and pretend it's a session cookie. -## - Domain-matching: Netscape uses the 2-dot rule for _all_ domains, not -## just the 7 special TLD's listed in their spec. And folks rely on -## that... - - def test_domain_return_ok(self): - # test optimization: .domain_return_ok() should filter out most - # domains in the CookieJar before we try to access them (because that - # may require disk access -- in particular, with MSIECookieJar) - # This is only a rough check for performance reasons, so it's not too - # critical as long as it's sufficiently liberal. - import cookielib, urllib2 - pol = cookielib.DefaultCookiePolicy() - for url, domain, ok in [ - ("http://foo.bar.com/", "blah.com", False), - ("http://foo.bar.com/", "rhubarb.blah.com", False), - ("http://foo.bar.com/", "rhubarb.foo.bar.com", False), - ("http://foo.bar.com/", ".foo.bar.com", True), - ("http://foo.bar.com/", "foo.bar.com", True), - ("http://foo.bar.com/", ".bar.com", True), - ("http://foo.bar.com/", "com", True), - ("http://foo.com/", "rhubarb.foo.com", False), - ("http://foo.com/", ".foo.com", True), - ("http://foo.com/", "foo.com", True), - ("http://foo.com/", "com", True), - ("http://foo/", "rhubarb.foo", False), - ("http://foo/", ".foo", True), - ("http://foo/", "foo", True), - ("http://foo/", "foo.local", True), - ("http://foo/", ".local", True), - ]: - request = urllib2.Request(url) - r = pol.domain_return_ok(domain, request) - if ok: self.assert_(r) - else: self.assert_(not r) - - def test_missing_value(self): - from cookielib import MozillaCookieJar, lwp_cookie_str - - # missing = sign in Cookie: header is regarded by Mozilla as a missing - # name, and by cookielib as a missing value - filename = support.TESTFN - c = MozillaCookieJar(filename) - interact_netscape(c, "http://www.acme.com/", 'eggs') - interact_netscape(c, "http://www.acme.com/", '"spam"; path=/foo/') - cookie = c._cookies["www.acme.com"]["/"]["eggs"] - self.assert_(cookie.value is None) - self.assertEquals(cookie.name, "eggs") - cookie = c._cookies["www.acme.com"]['/foo/']['"spam"'] - self.assert_(cookie.value is None) - self.assertEquals(cookie.name, '"spam"') - self.assertEquals(lwp_cookie_str(cookie), ( - r'"spam"; path="/foo/"; domain="www.acme.com"; ' - 'path_spec; discard; version=0')) - old_str = repr(c) - c.save(ignore_expires=True, ignore_discard=True) - try: - c = MozillaCookieJar(filename) - c.revert(ignore_expires=True, ignore_discard=True) - finally: - os.unlink(c.filename) - # cookies unchanged apart from lost info re. whether path was specified - self.assertEquals( - repr(c), - re.sub("path_specified=%s" % True, "path_specified=%s" % False, - old_str) - ) - self.assertEquals(interact_netscape(c, "http://www.acme.com/foo/"), - '"spam"; eggs') - - def test_rfc2109_handling(self): - # RFC 2109 cookies are handled as RFC 2965 or Netscape cookies, - # dependent on policy settings - from cookielib import CookieJar, DefaultCookiePolicy - - for rfc2109_as_netscape, rfc2965, version in [ - # default according to rfc2965 if not explicitly specified - (None, False, 0), - (None, True, 1), - # explicit rfc2109_as_netscape - (False, False, None), # version None here means no cookie stored - (False, True, 1), - (True, False, 0), - (True, True, 0), - ]: - policy = DefaultCookiePolicy( - rfc2109_as_netscape=rfc2109_as_netscape, - rfc2965=rfc2965) - c = CookieJar(policy) - interact_netscape(c, "http://www.example.com/", "ni=ni; Version=1") - try: - cookie = c._cookies["www.example.com"]["/"]["ni"] - except KeyError: - self.assert_(version is None) # didn't expect a stored cookie - else: - self.assertEqual(cookie.version, version) - # 2965 cookies are unaffected - interact_2965(c, "http://www.example.com/", - "foo=bar; Version=1") - if rfc2965: - cookie2965 = c._cookies["www.example.com"]["/"]["foo"] - self.assertEqual(cookie2965.version, 1) - - def test_ns_parser(self): - from cookielib import CookieJar, DEFAULT_HTTP_PORT - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/", - 'spam=eggs; DoMain=.acme.com; port; blArgh="feep"') - interact_netscape(c, "http://www.acme.com/", 'ni=ni; port=80,8080') - interact_netscape(c, "http://www.acme.com:80/", 'nini=ni') - interact_netscape(c, "http://www.acme.com:80/", 'foo=bar; expires=') - interact_netscape(c, "http://www.acme.com:80/", 'spam=eggs; ' - 'expires="Foo Bar 25 33:22:11 3022"') - - cookie = c._cookies[".acme.com"]["/"]["spam"] - self.assertEquals(cookie.domain, ".acme.com") - self.assert_(cookie.domain_specified) - self.assertEquals(cookie.port, DEFAULT_HTTP_PORT) - self.assert_(not cookie.port_specified) - # case is preserved - self.assert_(cookie.has_nonstandard_attr("blArgh") and - not cookie.has_nonstandard_attr("blargh")) - - cookie = c._cookies["www.acme.com"]["/"]["ni"] - self.assertEquals(cookie.domain, "www.acme.com") - self.assert_(not cookie.domain_specified) - self.assertEquals(cookie.port, "80,8080") - self.assert_(cookie.port_specified) - - cookie = c._cookies["www.acme.com"]["/"]["nini"] - self.assert_(cookie.port is None) - self.assert_(not cookie.port_specified) - - # invalid expires should not cause cookie to be dropped - foo = c._cookies["www.acme.com"]["/"]["foo"] - spam = c._cookies["www.acme.com"]["/"]["foo"] - self.assert_(foo.expires is None) - self.assert_(spam.expires is None) - - def test_ns_parser_special_names(self): - # names such as 'expires' are not special in first name=value pair - # of Set-Cookie: header - from cookielib import CookieJar - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/", 'expires=eggs') - interact_netscape(c, "http://www.acme.com/", 'version=eggs; spam=eggs') - - cookies = c._cookies["www.acme.com"]["/"] - self.assert_('expires' in cookies) - self.assert_('version' in cookies) - - def test_expires(self): - from cookielib import time2netscape, CookieJar - - # if expires is in future, keep cookie... - c = CookieJar() - future = time2netscape(time.time()+3600) - interact_netscape(c, "http://www.acme.com/", 'spam="bar"; expires=%s' % - future) - self.assertEquals(len(c), 1) - now = time2netscape(time.time()-1) - # ... and if in past or present, discard it - interact_netscape(c, "http://www.acme.com/", 'foo="eggs"; expires=%s' % - now) - h = interact_netscape(c, "http://www.acme.com/") - self.assertEquals(len(c), 1) - self.assert_('spam="bar"' in h and "foo" not in h) - - # max-age takes precedence over expires, and zero max-age is request to - # delete both new cookie and any old matching cookie - interact_netscape(c, "http://www.acme.com/", 'eggs="bar"; expires=%s' % - future) - interact_netscape(c, "http://www.acme.com/", 'bar="bar"; expires=%s' % - future) - self.assertEquals(len(c), 3) - interact_netscape(c, "http://www.acme.com/", 'eggs="bar"; ' - 'expires=%s; max-age=0' % future) - interact_netscape(c, "http://www.acme.com/", 'bar="bar"; ' - 'max-age=0; expires=%s' % future) - h = interact_netscape(c, "http://www.acme.com/") - self.assertEquals(len(c), 1) - - # test expiry at end of session for cookies with no expires attribute - interact_netscape(c, "http://www.rhubarb.net/", 'whum="fizz"') - self.assertEquals(len(c), 2) - c.clear_session_cookies() - self.assertEquals(len(c), 1) - self.assert_('spam="bar"' in h) - - # XXX RFC 2965 expiry rules (some apply to V0 too) - - def test_default_path(self): - from cookielib import CookieJar, DefaultCookiePolicy - - # RFC 2965 - pol = DefaultCookiePolicy(rfc2965=True) - - c = CookieJar(pol) - interact_2965(c, "http://www.acme.com/", 'spam="bar"; Version="1"') - self.assert_("/" in c._cookies["www.acme.com"]) - - c = CookieJar(pol) - interact_2965(c, "http://www.acme.com/blah", 'eggs="bar"; Version="1"') - self.assert_("/" in c._cookies["www.acme.com"]) - - c = CookieJar(pol) - interact_2965(c, "http://www.acme.com/blah/rhubarb", - 'eggs="bar"; Version="1"') - self.assert_("/blah/" in c._cookies["www.acme.com"]) - - c = CookieJar(pol) - interact_2965(c, "http://www.acme.com/blah/rhubarb/", - 'eggs="bar"; Version="1"') - self.assert_("/blah/rhubarb/" in c._cookies["www.acme.com"]) - - # Netscape - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/", 'spam="bar"') - self.assert_("/" in c._cookies["www.acme.com"]) - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/blah", 'eggs="bar"') - self.assert_("/" in c._cookies["www.acme.com"]) - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/blah/rhubarb", 'eggs="bar"') - self.assert_("/blah" in c._cookies["www.acme.com"]) - - c = CookieJar() - interact_netscape(c, "http://www.acme.com/blah/rhubarb/", 'eggs="bar"') - self.assert_("/blah/rhubarb" in c._cookies["www.acme.com"]) - - def test_escape_path(self): - from cookielib import escape_path - cases = [ - # quoted safe - ("/foo%2f/bar", "/foo%2F/bar"), - ("/foo%2F/bar", "/foo%2F/bar"), - # quoted % - ("/foo%%/bar", "/foo%%/bar"), - # quoted unsafe - ("/fo%19o/bar", "/fo%19o/bar"), - ("/fo%7do/bar", "/fo%7Do/bar"), - # unquoted safe - ("/foo/bar&", "/foo/bar&"), - ("/foo//bar", "/foo//bar"), - ("\176/foo/bar", "\176/foo/bar"), - # unquoted unsafe - ("/foo\031/bar", "/foo%19/bar"), - ("/\175foo/bar", "/%7Dfoo/bar"), - # unicode - ("/foo/bar\uabcd", "/foo/bar%EA%AF%8D"), # UTF-8 encoded - ] - for arg, result in cases: - self.assertEquals(escape_path(arg), result) - - def test_request_path(self): - from urllib2 import Request - from cookielib import request_path - # with parameters - req = Request("http://www.example.com/rheum/rhaponicum;" - "foo=bar;sing=song?apples=pears&spam=eggs#ni") - self.assertEquals(request_path(req), "/rheum/rhaponicum;" - "foo=bar;sing=song?apples=pears&spam=eggs#ni") - # without parameters - req = Request("http://www.example.com/rheum/rhaponicum?" - "apples=pears&spam=eggs#ni") - self.assertEquals(request_path(req), "/rheum/rhaponicum?" - "apples=pears&spam=eggs#ni") - # missing final slash - req = Request("http://www.example.com") - self.assertEquals(request_path(req), "/") - - def test_request_port(self): - from urllib2 import Request - from cookielib import request_port, DEFAULT_HTTP_PORT - req = Request("http://www.acme.com:1234/", - headers={"Host": "www.acme.com:4321"}) - self.assertEquals(request_port(req), "1234") - req = Request("http://www.acme.com/", - headers={"Host": "www.acme.com:4321"}) - self.assertEquals(request_port(req), DEFAULT_HTTP_PORT) - - def test_request_host(self): - from urllib2 import Request - from cookielib import request_host - # this request is illegal (RFC2616, 14.2.3) - req = Request("http://1.1.1.1/", - headers={"Host": "www.acme.com:80"}) - # libwww-perl wants this response, but that seems wrong (RFC 2616, - # section 5.2, point 1., and RFC 2965 section 1, paragraph 3) - #self.assertEquals(request_host(req), "www.acme.com") - self.assertEquals(request_host(req), "1.1.1.1") - req = Request("http://www.acme.com/", - headers={"Host": "irrelevant.com"}) - self.assertEquals(request_host(req), "www.acme.com") - # not actually sure this one is valid Request object, so maybe should - # remove test for no host in url in request_host function? - req = Request("/resource.html", - headers={"Host": "www.acme.com"}) - self.assertEquals(request_host(req), "www.acme.com") - # port shouldn't be in request-host - req = Request("http://www.acme.com:2345/resource.html", - headers={"Host": "www.acme.com:5432"}) - self.assertEquals(request_host(req), "www.acme.com") - - def test_is_HDN(self): - from cookielib import is_HDN - self.assert_(is_HDN("foo.bar.com")) - self.assert_(is_HDN("1foo2.3bar4.5com")) - self.assert_(not is_HDN("192.168.1.1")) - self.assert_(not is_HDN("")) - self.assert_(not is_HDN(".")) - self.assert_(not is_HDN(".foo.bar.com")) - self.assert_(not is_HDN("..foo")) - self.assert_(not is_HDN("foo.")) - - def test_reach(self): - from cookielib import reach - self.assertEquals(reach("www.acme.com"), ".acme.com") - self.assertEquals(reach("acme.com"), "acme.com") - self.assertEquals(reach("acme.local"), ".local") - self.assertEquals(reach(".local"), ".local") - self.assertEquals(reach(".com"), ".com") - self.assertEquals(reach("."), ".") - self.assertEquals(reach(""), "") - self.assertEquals(reach("192.168.0.1"), "192.168.0.1") - - def test_domain_match(self): - from cookielib import domain_match, user_domain_match - self.assert_(domain_match("192.168.1.1", "192.168.1.1")) - self.assert_(not domain_match("192.168.1.1", ".168.1.1")) - self.assert_(domain_match("x.y.com", "x.Y.com")) - self.assert_(domain_match("x.y.com", ".Y.com")) - self.assert_(not domain_match("x.y.com", "Y.com")) - self.assert_(domain_match("a.b.c.com", ".c.com")) - self.assert_(not domain_match(".c.com", "a.b.c.com")) - self.assert_(domain_match("example.local", ".local")) - self.assert_(not domain_match("blah.blah", "")) - self.assert_(not domain_match("", ".rhubarb.rhubarb")) - self.assert_(domain_match("", "")) - - self.assert_(user_domain_match("acme.com", "acme.com")) - self.assert_(not user_domain_match("acme.com", ".acme.com")) - self.assert_(user_domain_match("rhubarb.acme.com", ".acme.com")) - self.assert_(user_domain_match("www.rhubarb.acme.com", ".acme.com")) - self.assert_(user_domain_match("x.y.com", "x.Y.com")) - self.assert_(user_domain_match("x.y.com", ".Y.com")) - self.assert_(not user_domain_match("x.y.com", "Y.com")) - self.assert_(user_domain_match("y.com", "Y.com")) - self.assert_(not user_domain_match(".y.com", "Y.com")) - self.assert_(user_domain_match(".y.com", ".Y.com")) - self.assert_(user_domain_match("x.y.com", ".com")) - self.assert_(not user_domain_match("x.y.com", "com")) - self.assert_(not user_domain_match("x.y.com", "m")) - self.assert_(not user_domain_match("x.y.com", ".m")) - self.assert_(not user_domain_match("x.y.com", "")) - self.assert_(not user_domain_match("x.y.com", ".")) - self.assert_(user_domain_match("192.168.1.1", "192.168.1.1")) - # not both HDNs, so must string-compare equal to match - self.assert_(not user_domain_match("192.168.1.1", ".168.1.1")) - self.assert_(not user_domain_match("192.168.1.1", ".")) - # empty string is a special case - self.assert_(not user_domain_match("192.168.1.1", "")) - - def test_wrong_domain(self): - # Cookies whose effective request-host name does not domain-match the - # domain are rejected. - - # XXX far from complete - from cookielib import CookieJar - c = CookieJar() - interact_2965(c, "http://www.nasty.com/", - 'foo=bar; domain=friendly.org; Version="1"') - self.assertEquals(len(c), 0) - - def test_strict_domain(self): - # Cookies whose domain is a country-code tld like .co.uk should - # not be set if CookiePolicy.strict_domain is true. - from cookielib import CookieJar, DefaultCookiePolicy - - cp = DefaultCookiePolicy(strict_domain=True) - cj = CookieJar(policy=cp) - interact_netscape(cj, "http://example.co.uk/", 'no=problemo') - interact_netscape(cj, "http://example.co.uk/", - 'okey=dokey; Domain=.example.co.uk') - self.assertEquals(len(cj), 2) - for pseudo_tld in [".co.uk", ".org.za", ".tx.us", ".name.us"]: - interact_netscape(cj, "http://example.%s/" % pseudo_tld, - 'spam=eggs; Domain=.co.uk') - self.assertEquals(len(cj), 2) - - def test_two_component_domain_ns(self): - # Netscape: .www.bar.com, www.bar.com, .bar.com, bar.com, no domain - # should all get accepted, as should .acme.com, acme.com and no domain - # for 2-component domains like acme.com. - from cookielib import CookieJar, DefaultCookiePolicy - - c = CookieJar() - - # two-component V0 domain is OK - interact_netscape(c, "http://foo.net/", 'ns=bar') - self.assertEquals(len(c), 1) - self.assertEquals(c._cookies["foo.net"]["/"]["ns"].value, "bar") - self.assertEquals(interact_netscape(c, "http://foo.net/"), "ns=bar") - # *will* be returned to any other domain (unlike RFC 2965)... - self.assertEquals(interact_netscape(c, "http://www.foo.net/"), - "ns=bar") - # ...unless requested otherwise - pol = DefaultCookiePolicy( - strict_ns_domain=DefaultCookiePolicy.DomainStrictNonDomain) - c.set_policy(pol) - self.assertEquals(interact_netscape(c, "http://www.foo.net/"), "") - - # unlike RFC 2965, even explicit two-component domain is OK, - # because .foo.net matches foo.net - interact_netscape(c, "http://foo.net/foo/", - 'spam1=eggs; domain=foo.net') - # even if starts with a dot -- in NS rules, .foo.net matches foo.net! - interact_netscape(c, "http://foo.net/foo/bar/", - 'spam2=eggs; domain=.foo.net') - self.assertEquals(len(c), 3) - self.assertEquals(c._cookies[".foo.net"]["/foo"]["spam1"].value, - "eggs") - self.assertEquals(c._cookies[".foo.net"]["/foo/bar"]["spam2"].value, - "eggs") - self.assertEquals(interact_netscape(c, "http://foo.net/foo/bar/"), - "spam2=eggs; spam1=eggs; ns=bar") - - # top-level domain is too general - interact_netscape(c, "http://foo.net/", 'nini="ni"; domain=.net') - self.assertEquals(len(c), 3) - -## # Netscape protocol doesn't allow non-special top level domains (such -## # as co.uk) in the domain attribute unless there are at least three -## # dots in it. - # Oh yes it does! Real implementations don't check this, and real - # cookies (of course) rely on that behaviour. - interact_netscape(c, "http://foo.co.uk", 'nasty=trick; domain=.co.uk') -## self.assertEquals(len(c), 2) - self.assertEquals(len(c), 4) - - def test_two_component_domain_rfc2965(self): - from cookielib import CookieJar, DefaultCookiePolicy - - pol = DefaultCookiePolicy(rfc2965=True) - c = CookieJar(pol) - - # two-component V1 domain is OK - interact_2965(c, "http://foo.net/", 'foo=bar; Version="1"') - self.assertEquals(len(c), 1) - self.assertEquals(c._cookies["foo.net"]["/"]["foo"].value, "bar") - self.assertEquals(interact_2965(c, "http://foo.net/"), - "$Version=1; foo=bar") - # won't be returned to any other domain (because domain was implied) - self.assertEquals(interact_2965(c, "http://www.foo.net/"), "") - - # unless domain is given explicitly, because then it must be - # rewritten to start with a dot: foo.net --> .foo.net, which does - # not domain-match foo.net - interact_2965(c, "http://foo.net/foo", - 'spam=eggs; domain=foo.net; path=/foo; Version="1"') - self.assertEquals(len(c), 1) - self.assertEquals(interact_2965(c, "http://foo.net/foo"), - "$Version=1; foo=bar") - - # explicit foo.net from three-component domain www.foo.net *does* get - # set, because .foo.net domain-matches .foo.net - interact_2965(c, "http://www.foo.net/foo/", - 'spam=eggs; domain=foo.net; Version="1"') - self.assertEquals(c._cookies[".foo.net"]["/foo/"]["spam"].value, - "eggs") - self.assertEquals(len(c), 2) - self.assertEquals(interact_2965(c, "http://foo.net/foo/"), - "$Version=1; foo=bar") - self.assertEquals(interact_2965(c, "http://www.foo.net/foo/"), - '$Version=1; spam=eggs; $Domain="foo.net"') - - # top-level domain is too general - interact_2965(c, "http://foo.net/", - 'ni="ni"; domain=".net"; Version="1"') - self.assertEquals(len(c), 2) - - # RFC 2965 doesn't require blocking this - interact_2965(c, "http://foo.co.uk/", - 'nasty=trick; domain=.co.uk; Version="1"') - self.assertEquals(len(c), 3) - - def test_domain_allow(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - - c = CookieJar(policy=DefaultCookiePolicy( - blocked_domains=["acme.com"], - allowed_domains=["www.acme.com"])) - - req = Request("http://acme.com/") - headers = ["Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/"] - res = FakeResponse(headers, "http://acme.com/") - c.extract_cookies(res, req) - self.assertEquals(len(c), 0) - - req = Request("http://www.acme.com/") - res = FakeResponse(headers, "http://www.acme.com/") - c.extract_cookies(res, req) - self.assertEquals(len(c), 1) - - req = Request("http://www.coyote.com/") - res = FakeResponse(headers, "http://www.coyote.com/") - c.extract_cookies(res, req) - self.assertEquals(len(c), 1) - - # set a cookie with non-allowed domain... - req = Request("http://www.coyote.com/") - res = FakeResponse(headers, "http://www.coyote.com/") - cookies = c.make_cookies(res, req) - c.set_cookie(cookies[0]) - self.assertEquals(len(c), 2) - # ... and check is doesn't get returned - c.add_cookie_header(req) - self.assert_(not req.has_header("Cookie")) - - def test_domain_block(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - - pol = DefaultCookiePolicy( - rfc2965=True, blocked_domains=[".acme.com"]) - c = CookieJar(policy=pol) - headers = ["Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/"] - - req = Request("http://www.acme.com/") - res = FakeResponse(headers, "http://www.acme.com/") - c.extract_cookies(res, req) - self.assertEquals(len(c), 0) - - p = pol.set_blocked_domains(["acme.com"]) - c.extract_cookies(res, req) - self.assertEquals(len(c), 1) - - c.clear() - req = Request("http://www.roadrunner.net/") - res = FakeResponse(headers, "http://www.roadrunner.net/") - c.extract_cookies(res, req) - self.assertEquals(len(c), 1) - req = Request("http://www.roadrunner.net/") - c.add_cookie_header(req) - self.assert_((req.has_header("Cookie") and - req.has_header("Cookie2"))) - - c.clear() - pol.set_blocked_domains([".acme.com"]) - c.extract_cookies(res, req) - self.assertEquals(len(c), 1) - - # set a cookie with blocked domain... - req = Request("http://www.acme.com/") - res = FakeResponse(headers, "http://www.acme.com/") - cookies = c.make_cookies(res, req) - c.set_cookie(cookies[0]) - self.assertEquals(len(c), 2) - # ... and check is doesn't get returned - c.add_cookie_header(req) - self.assert_(not req.has_header("Cookie")) - - def test_secure(self): - from cookielib import CookieJar, DefaultCookiePolicy - - for ns in True, False: - for whitespace in " ", "": - c = CookieJar() - if ns: - pol = DefaultCookiePolicy(rfc2965=False) - int = interact_netscape - vs = "" - else: - pol = DefaultCookiePolicy(rfc2965=True) - int = interact_2965 - vs = "; Version=1" - c.set_policy(pol) - url = "http://www.acme.com/" - int(c, url, "foo1=bar%s%s" % (vs, whitespace)) - int(c, url, "foo2=bar%s; secure%s" % (vs, whitespace)) - self.assert_( - not c._cookies["www.acme.com"]["/"]["foo1"].secure, - "non-secure cookie registered secure") - self.assert_( - c._cookies["www.acme.com"]["/"]["foo2"].secure, - "secure cookie registered non-secure") - - def test_quote_cookie_value(self): - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar(policy=DefaultCookiePolicy(rfc2965=True)) - interact_2965(c, "http://www.acme.com/", r'foo=\b"a"r; Version=1') - h = interact_2965(c, "http://www.acme.com/") - self.assertEquals(h, r'$Version=1; foo=\\b\"a\"r') - - def test_missing_final_slash(self): - # Missing slash from request URL's abs_path should be assumed present. - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - url = "http://www.acme.com" - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) - interact_2965(c, url, "foo=bar; Version=1") - req = Request(url) - self.assertEquals(len(c), 1) - c.add_cookie_header(req) - self.assert_(req.has_header("Cookie")) - - def test_domain_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - - pol = DefaultCookiePolicy(rfc2965=True) - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, "spam=eggs; Version=1") - h = interact_2965(c, url) - self.assert_("Domain" not in h, - "absent domain returned with domain present") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, 'spam=eggs; Version=1; Domain=.bar.com') - h = interact_2965(c, url) - self.assert_('$Domain=".bar.com"' in h, "domain not returned") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - # note missing initial dot in Domain - interact_2965(c, url, 'spam=eggs; Version=1; Domain=bar.com') - h = interact_2965(c, url) - self.assert_('$Domain="bar.com"' in h, "domain not returned") - - def test_path_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - - pol = DefaultCookiePolicy(rfc2965=True) - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, "spam=eggs; Version=1") - h = interact_2965(c, url) - self.assert_("Path" not in h, - "absent path returned with path present") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, 'spam=eggs; Version=1; Path=/') - h = interact_2965(c, url) - self.assert_('$Path="/"' in h, "path not returned") - - def test_port_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - - pol = DefaultCookiePolicy(rfc2965=True) - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, "spam=eggs; Version=1") - h = interact_2965(c, url) - self.assert_("Port" not in h, - "absent port returned with port present") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, "spam=eggs; Version=1; Port") - h = interact_2965(c, url) - self.assert_(re.search("\$Port([^=]|$)", h), - "port with no value not returned with no value") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, 'spam=eggs; Version=1; Port="80"') - h = interact_2965(c, url) - self.assert_('$Port="80"' in h, - "port with single value not returned with single value") - - c = CookieJar(pol) - url = "http://foo.bar.com/" - interact_2965(c, url, 'spam=eggs; Version=1; Port="80,8080"') - h = interact_2965(c, url) - self.assert_('$Port="80,8080"' in h, - "port with multiple values not returned with multiple " - "values") - - def test_no_return_comment(self): - from cookielib import CookieJar, DefaultCookiePolicy - - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) - url = "http://foo.bar.com/" - interact_2965(c, url, 'spam=eggs; Version=1; ' - 'Comment="does anybody read these?"; ' - 'CommentURL="http://foo.bar.net/comment.html"') - h = interact_2965(c, url) - self.assert_( - "Comment" not in h, - "Comment or CommentURL cookie-attributes returned to server") - - def test_Cookie_iterator(self): - from cookielib import CookieJar, Cookie, DefaultCookiePolicy - - cs = CookieJar(DefaultCookiePolicy(rfc2965=True)) - # add some random cookies - interact_2965(cs, "http://blah.spam.org/", 'foo=eggs; Version=1; ' - 'Comment="does anybody read these?"; ' - 'CommentURL="http://foo.bar.net/comment.html"') - interact_netscape(cs, "http://www.acme.com/blah/", "spam=bar; secure") - interact_2965(cs, "http://www.acme.com/blah/", - "foo=bar; secure; Version=1") - interact_2965(cs, "http://www.acme.com/blah/", - "foo=bar; path=/; Version=1") - interact_2965(cs, "http://www.sol.no", - r'bang=wallop; version=1; domain=".sol.no"; ' - r'port="90,100, 80,8080"; ' - r'max-age=100; Comment = "Just kidding! (\"|\\\\) "') - - versions = [1, 1, 1, 0, 1] - names = ["bang", "foo", "foo", "spam", "foo"] - domains = [".sol.no", "blah.spam.org", "www.acme.com", - "www.acme.com", "www.acme.com"] - paths = ["/", "/", "/", "/blah", "/blah/"] - - for i in range(4): - i = 0 - for c in cs: - self.assert_(isinstance(c, Cookie)) - self.assertEquals(c.version, versions[i]) - self.assertEquals(c.name, names[i]) - self.assertEquals(c.domain, domains[i]) - self.assertEquals(c.path, paths[i]) - i = i + 1 - - def test_parse_ns_headers(self): - from cookielib import parse_ns_headers - - # missing domain value (invalid cookie) - self.assertEquals( - parse_ns_headers(["foo=bar; path=/; domain"]), - [[("foo", "bar"), - ("path", "/"), ("domain", None), ("version", "0")]] - ) - # invalid expires value - self.assertEquals( - parse_ns_headers(["foo=bar; expires=Foo Bar 12 33:22:11 2000"]), - [[("foo", "bar"), ("expires", None), ("version", "0")]] - ) - # missing cookie value (valid cookie) - self.assertEquals( - parse_ns_headers(["foo"]), - [[("foo", None), ("version", "0")]] - ) - # shouldn't add version if header is empty - self.assertEquals(parse_ns_headers([""]), []) - - def test_bad_cookie_header(self): - - def cookiejar_from_cookie_headers(headers): - from cookielib import CookieJar - from urllib2 import Request - c = CookieJar() - req = Request("http://www.example.com/") - r = FakeResponse(headers, "http://www.example.com/") - c.extract_cookies(r, req) - return c - - # none of these bad headers should cause an exception to be raised - for headers in [ - ["Set-Cookie: "], # actually, nothing wrong with this - ["Set-Cookie2: "], # ditto - # missing domain value - ["Set-Cookie2: a=foo; path=/; Version=1; domain"], - # bad max-age - ["Set-Cookie: b=foo; max-age=oops"], - ]: - c = cookiejar_from_cookie_headers(headers) - # these bad cookies shouldn't be set - self.assertEquals(len(c), 0) - - # cookie with invalid expires is treated as session cookie - headers = ["Set-Cookie: c=foo; expires=Foo Bar 12 33:22:11 2000"] - c = cookiejar_from_cookie_headers(headers) - cookie = c._cookies["www.example.com"]["/"]["c"] - self.assert_(cookie.expires is None) - - -class LWPCookieTests(TestCase): - # Tests taken from libwww-perl, with a few modifications and additions. - - def test_netscape_example_1(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - - #------------------------------------------------------------------- - # First we check that it works for the original example at - # http://www.netscape.com/newsref/std/cookie_spec.html - - # Client requests a document, and receives in the response: - # - # Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/; expires=Wednesday, 09-Nov-99 23:12:40 GMT - # - # When client requests a URL in path "/" on this server, it sends: - # - # Cookie: CUSTOMER=WILE_E_COYOTE - # - # Client requests a document, and receives in the response: - # - # Set-Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001; path=/ - # - # When client requests a URL in path "/" on this server, it sends: - # - # Cookie: CUSTOMER=WILE_E_COYOTE; PART_NUMBER=ROCKET_LAUNCHER_0001 - # - # Client receives: - # - # Set-Cookie: SHIPPING=FEDEX; path=/fo - # - # When client requests a URL in path "/" on this server, it sends: - # - # Cookie: CUSTOMER=WILE_E_COYOTE; PART_NUMBER=ROCKET_LAUNCHER_0001 - # - # When client requests a URL in path "/foo" on this server, it sends: - # - # Cookie: CUSTOMER=WILE_E_COYOTE; PART_NUMBER=ROCKET_LAUNCHER_0001; SHIPPING=FEDEX - # - # The last Cookie is buggy, because both specifications say that the - # most specific cookie must be sent first. SHIPPING=FEDEX is the - # most specific and should thus be first. - - year_plus_one = time.localtime()[0] + 1 - - headers = [] - - c = CookieJar(DefaultCookiePolicy(rfc2965 = True)) - - #req = Request("http://1.1.1.1/", - # headers={"Host": "www.acme.com:80"}) - req = Request("http://www.acme.com:80/", - headers={"Host": "www.acme.com:80"}) - - headers.append( - "Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/ ; " - "expires=Wednesday, 09-Nov-%d 23:12:40 GMT" % year_plus_one) - res = FakeResponse(headers, "http://www.acme.com/") - c.extract_cookies(res, req) - - req = Request("http://www.acme.com/") - c.add_cookie_header(req) - - self.assertEqual(req.get_header("Cookie"), "CUSTOMER=WILE_E_COYOTE") - self.assertEqual(req.get_header("Cookie2"), '$Version="1"') - - headers.append("Set-Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001; path=/") - res = FakeResponse(headers, "http://www.acme.com/") - c.extract_cookies(res, req) - - req = Request("http://www.acme.com/foo/bar") - c.add_cookie_header(req) - - h = req.get_header("Cookie") - self.assert_("PART_NUMBER=ROCKET_LAUNCHER_0001" in h and - "CUSTOMER=WILE_E_COYOTE" in h) - - headers.append('Set-Cookie: SHIPPING=FEDEX; path=/foo') - res = FakeResponse(headers, "http://www.acme.com") - c.extract_cookies(res, req) - - req = Request("http://www.acme.com/") - c.add_cookie_header(req) - - h = req.get_header("Cookie") - self.assert_("PART_NUMBER=ROCKET_LAUNCHER_0001" in h and - "CUSTOMER=WILE_E_COYOTE" in h and - "SHIPPING=FEDEX" not in h) - - req = Request("http://www.acme.com/foo/") - c.add_cookie_header(req) - - h = req.get_header("Cookie") - self.assert_(("PART_NUMBER=ROCKET_LAUNCHER_0001" in h and - "CUSTOMER=WILE_E_COYOTE" in h and - h.startswith("SHIPPING=FEDEX;"))) - - def test_netscape_example_2(self): - from cookielib import CookieJar - from urllib2 import Request - - # Second Example transaction sequence: - # - # Assume all mappings from above have been cleared. - # - # Client receives: - # - # Set-Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001; path=/ - # - # When client requests a URL in path "/" on this server, it sends: - # - # Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001 - # - # Client receives: - # - # Set-Cookie: PART_NUMBER=RIDING_ROCKET_0023; path=/ammo - # - # When client requests a URL in path "/ammo" on this server, it sends: - # - # Cookie: PART_NUMBER=RIDING_ROCKET_0023; PART_NUMBER=ROCKET_LAUNCHER_0001 - # - # NOTE: There are two name/value pairs named "PART_NUMBER" due to - # the inheritance of the "/" mapping in addition to the "/ammo" mapping. - - c = CookieJar() - headers = [] - - req = Request("http://www.acme.com/") - headers.append("Set-Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001; path=/") - res = FakeResponse(headers, "http://www.acme.com/") - - c.extract_cookies(res, req) - - req = Request("http://www.acme.com/") - c.add_cookie_header(req) - - self.assertEquals(req.get_header("Cookie"), - "PART_NUMBER=ROCKET_LAUNCHER_0001") - - headers.append( - "Set-Cookie: PART_NUMBER=RIDING_ROCKET_0023; path=/ammo") - res = FakeResponse(headers, "http://www.acme.com/") - c.extract_cookies(res, req) - - req = Request("http://www.acme.com/ammo") - c.add_cookie_header(req) - - self.assert_(re.search(r"PART_NUMBER=RIDING_ROCKET_0023;\s*" - "PART_NUMBER=ROCKET_LAUNCHER_0001", - req.get_header("Cookie"))) - - def test_ietf_example_1(self): - from cookielib import CookieJar, DefaultCookiePolicy - #------------------------------------------------------------------- - # Then we test with the examples from draft-ietf-http-state-man-mec-03.txt - # - # 5. EXAMPLES - - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) - - # - # 5.1 Example 1 - # - # Most detail of request and response headers has been omitted. Assume - # the user agent has no stored cookies. - # - # 1. User Agent -> Server - # - # POST /acme/login HTTP/1.1 - # [form data] - # - # User identifies self via a form. - # - # 2. Server -> User Agent - # - # HTTP/1.1 200 OK - # Set-Cookie2: Customer="WILE_E_COYOTE"; Version="1"; Path="/acme" - # - # Cookie reflects user's identity. - - cookie = interact_2965( - c, 'http://www.acme.com/acme/login', - 'Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"') - self.assert_(not cookie) - - # - # 3. User Agent -> Server - # - # POST /acme/pickitem HTTP/1.1 - # Cookie: $Version="1"; Customer="WILE_E_COYOTE"; $Path="/acme" - # [form data] - # - # User selects an item for ``shopping basket.'' - # - # 4. Server -> User Agent - # - # HTTP/1.1 200 OK - # Set-Cookie2: Part_Number="Rocket_Launcher_0001"; Version="1"; - # Path="/acme" - # - # Shopping basket contains an item. - - cookie = interact_2965(c, 'http://www.acme.com/acme/pickitem', - 'Part_Number="Rocket_Launcher_0001"; ' - 'Version="1"; Path="/acme"'); - self.assert_(re.search( - r'^\$Version="?1"?; Customer="?WILE_E_COYOTE"?; \$Path="/acme"$', - cookie)) - - # - # 5. User Agent -> Server - # - # POST /acme/shipping HTTP/1.1 - # Cookie: $Version="1"; - # Customer="WILE_E_COYOTE"; $Path="/acme"; - # Part_Number="Rocket_Launcher_0001"; $Path="/acme" - # [form data] - # - # User selects shipping method from form. - # - # 6. Server -> User Agent - # - # HTTP/1.1 200 OK - # Set-Cookie2: Shipping="FedEx"; Version="1"; Path="/acme" - # - # New cookie reflects shipping method. - - cookie = interact_2965(c, "http://www.acme.com/acme/shipping", - 'Shipping="FedEx"; Version="1"; Path="/acme"') - - self.assert_(re.search(r'^\$Version="?1"?;', cookie)) - self.assert_(re.search(r'Part_Number="?Rocket_Launcher_0001"?;' - '\s*\$Path="\/acme"', cookie)) - self.assert_(re.search(r'Customer="?WILE_E_COYOTE"?;\s*\$Path="\/acme"', - cookie)) - - # - # 7. User Agent -> Server - # - # POST /acme/process HTTP/1.1 - # Cookie: $Version="1"; - # Customer="WILE_E_COYOTE"; $Path="/acme"; - # Part_Number="Rocket_Launcher_0001"; $Path="/acme"; - # Shipping="FedEx"; $Path="/acme" - # [form data] - # - # User chooses to process order. - # - # 8. Server -> User Agent - # - # HTTP/1.1 200 OK - # - # Transaction is complete. - - cookie = interact_2965(c, "http://www.acme.com/acme/process") - self.assert_( - re.search(r'Shipping="?FedEx"?;\s*\$Path="\/acme"', cookie) and - "WILE_E_COYOTE" in cookie) - - # - # The user agent makes a series of requests on the origin server, after - # each of which it receives a new cookie. All the cookies have the same - # Path attribute and (default) domain. Because the request URLs all have - # /acme as a prefix, and that matches the Path attribute, each request - # contains all the cookies received so far. - - def test_ietf_example_2(self): - from cookielib import CookieJar, DefaultCookiePolicy - - # 5.2 Example 2 - # - # This example illustrates the effect of the Path attribute. All detail - # of request and response headers has been omitted. Assume the user agent - # has no stored cookies. - - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) - - # Imagine the user agent has received, in response to earlier requests, - # the response headers - # - # Set-Cookie2: Part_Number="Rocket_Launcher_0001"; Version="1"; - # Path="/acme" - # - # and - # - # Set-Cookie2: Part_Number="Riding_Rocket_0023"; Version="1"; - # Path="/acme/ammo" - - interact_2965( - c, "http://www.acme.com/acme/ammo/specific", - 'Part_Number="Rocket_Launcher_0001"; Version="1"; Path="/acme"', - 'Part_Number="Riding_Rocket_0023"; Version="1"; Path="/acme/ammo"') - - # A subsequent request by the user agent to the (same) server for URLs of - # the form /acme/ammo/... would include the following request header: - # - # Cookie: $Version="1"; - # Part_Number="Riding_Rocket_0023"; $Path="/acme/ammo"; - # Part_Number="Rocket_Launcher_0001"; $Path="/acme" - # - # Note that the NAME=VALUE pair for the cookie with the more specific Path - # attribute, /acme/ammo, comes before the one with the less specific Path - # attribute, /acme. Further note that the same cookie name appears more - # than once. - - cookie = interact_2965(c, "http://www.acme.com/acme/ammo/...") - self.assert_( - re.search(r"Riding_Rocket_0023.*Rocket_Launcher_0001", cookie)) - - # A subsequent request by the user agent to the (same) server for a URL of - # the form /acme/parts/ would include the following request header: - # - # Cookie: $Version="1"; Part_Number="Rocket_Launcher_0001"; $Path="/acme" - # - # Here, the second cookie's Path attribute /acme/ammo is not a prefix of - # the request URL, /acme/parts/, so the cookie does not get forwarded to - # the server. - - cookie = interact_2965(c, "http://www.acme.com/acme/parts/") - self.assert_("Rocket_Launcher_0001" in cookie and - "Riding_Rocket_0023" not in cookie) - - def test_rejection(self): - # Test rejection of Set-Cookie2 responses based on domain, path, port. - from cookielib import DefaultCookiePolicy, LWPCookieJar - - pol = DefaultCookiePolicy(rfc2965=True) - - c = LWPCookieJar(policy=pol) - - max_age = "max-age=3600" - - # illegal domain (no embedded dots) - cookie = interact_2965(c, "http://www.acme.com", - 'foo=bar; domain=".com"; version=1') - self.assert_(not c) - - # legal domain - cookie = interact_2965(c, "http://www.acme.com", - 'ping=pong; domain="acme.com"; version=1') - self.assertEquals(len(c), 1) - - # illegal domain (host prefix "www.a" contains a dot) - cookie = interact_2965(c, "http://www.a.acme.com", - 'whiz=bang; domain="acme.com"; version=1') - self.assertEquals(len(c), 1) - - # legal domain - cookie = interact_2965(c, "http://www.a.acme.com", - 'wow=flutter; domain=".a.acme.com"; version=1') - self.assertEquals(len(c), 2) - - # can't partially match an IP-address - cookie = interact_2965(c, "http://125.125.125.125", - 'zzzz=ping; domain="125.125.125"; version=1') - self.assertEquals(len(c), 2) - - # illegal path (must be prefix of request path) - cookie = interact_2965(c, "http://www.sol.no", - 'blah=rhubarb; domain=".sol.no"; path="/foo"; ' - 'version=1') - self.assertEquals(len(c), 2) - - # legal path - cookie = interact_2965(c, "http://www.sol.no/foo/bar", - 'bing=bong; domain=".sol.no"; path="/foo"; ' - 'version=1') - self.assertEquals(len(c), 3) - - # illegal port (request-port not in list) - cookie = interact_2965(c, "http://www.sol.no", - 'whiz=ffft; domain=".sol.no"; port="90,100"; ' - 'version=1') - self.assertEquals(len(c), 3) - - # legal port - cookie = interact_2965( - c, "http://www.sol.no", - r'bang=wallop; version=1; domain=".sol.no"; ' - r'port="90,100, 80,8080"; ' - r'max-age=100; Comment = "Just kidding! (\"|\\\\) "') - self.assertEquals(len(c), 4) - - # port attribute without any value (current port) - cookie = interact_2965(c, "http://www.sol.no", - 'foo9=bar; version=1; domain=".sol.no"; port; ' - 'max-age=100;') - self.assertEquals(len(c), 5) - - # encoded path - # LWP has this test, but unescaping allowed path characters seems - # like a bad idea, so I think this should fail: -## cookie = interact_2965(c, "http://www.sol.no/foo/", -## r'foo8=bar; version=1; path="/%66oo"') - # but this is OK, because '<' is not an allowed HTTP URL path - # character: - cookie = interact_2965(c, "http://www.sol.no/ %s, %s, %s (%s)" % (s, t, t2, t3, test_t)) def test_http2time_garbage(self): - from cookielib import http2time - for test in [ '', 'Garbage', @@ -99,8 +98,6 @@ class HeaderTests(TestCase): def test_parse_ns_headers(self): - from cookielib import parse_ns_headers - # quotes should be stripped expected = [[('foo', 'bar'), ('expires', 2209069412), ('version', '0')]] for hdr in [ @@ -112,24 +109,18 @@ def test_parse_ns_headers_special_names(self): # names such as 'expires' are not special in first name=value pair # of Set-Cookie: header - from cookielib import parse_ns_headers - # Cookie with name 'expires' hdr = 'expires=01 Jan 2040 22:23:32 GMT' expected = [[("expires", "01 Jan 2040 22:23:32 GMT"), ("version", "0")]] self.assertEquals(parse_ns_headers([hdr]), expected) def test_join_header_words(self): - from cookielib import join_header_words - joined = join_header_words([[("foo", None), ("bar", "baz")]]) self.assertEquals(joined, "foo; bar=baz") self.assertEquals(join_header_words([[]]), "") def test_split_header_words(self): - from cookielib import split_header_words - tests = [ ("foo", [[("foo", None)]]), ("foo=bar", [[("foo", "bar")]]), @@ -164,8 +155,6 @@ """ % (arg, expect, result)) def test_roundtrip(self): - from cookielib import split_header_words, join_header_words - tests = [ ("foo", "foo"), ("foo=bar", "foo=bar"), @@ -218,8 +207,7 @@ def _interact(cookiejar, url, set_cookie_hdrs, hdr_name): """Perform a single request / response cycle, returning Cookie: header.""" - from urllib2 import Request - req = Request(url) + req = urllib2.Request(url) cookiejar.add_cookie_header(req) cookie_hdr = req.get_header("Cookie", "") headers = [] @@ -233,7 +221,6 @@ class FileCookieJarTests(TestCase): def test_lwp_valueless_cookie(self): # cookies with no value should be saved and loaded consistently - from cookielib import LWPCookieJar filename = support.TESTFN c = LWPCookieJar() interact_netscape(c, "http://www.acme.com/", 'boo') @@ -248,7 +235,6 @@ self.assertEqual(c._cookies["www.acme.com"]["/"]["boo"].value, None) def test_bad_magic(self): - from cookielib import LWPCookieJar, MozillaCookieJar, LoadError # IOErrors (eg. file doesn't exist) are allowed to propagate filename = support.TESTFN for cookiejar_class in LWPCookieJar, MozillaCookieJar: @@ -326,8 +312,7 @@ # may require disk access -- in particular, with MSIECookieJar) # This is only a rough check for performance reasons, so it's not too # critical as long as it's sufficiently liberal. - import cookielib, urllib2 - pol = cookielib.DefaultCookiePolicy() + pol = DefaultCookiePolicy() for url, domain, ok in [ ("http://foo.bar.com/", "blah.com", False), ("http://foo.bar.com/", "rhubarb.blah.com", False), @@ -352,10 +337,8 @@ else: self.assert_(not r) def test_missing_value(self): - from cookielib import MozillaCookieJar, lwp_cookie_str - # missing = sign in Cookie: header is regarded by Mozilla as a missing - # name, and by cookielib as a missing value + # name, and by http.cookiejar as a missing value filename = support.TESTFN c = MozillaCookieJar(filename) interact_netscape(c, "http://www.acme.com/", 'eggs') @@ -388,8 +371,6 @@ def test_rfc2109_handling(self): # RFC 2109 cookies are handled as RFC 2965 or Netscape cookies, # dependent on policy settings - from cookielib import CookieJar, DefaultCookiePolicy - for rfc2109_as_netscape, rfc2965, version in [ # default according to rfc2965 if not explicitly specified (None, False, 0), @@ -419,8 +400,6 @@ self.assertEqual(cookie2965.version, 1) def test_ns_parser(self): - from cookielib import CookieJar, DEFAULT_HTTP_PORT - c = CookieJar() interact_netscape(c, "http://www.acme.com/", 'spam=eggs; DoMain=.acme.com; port; blArgh="feep"') @@ -458,8 +437,6 @@ def test_ns_parser_special_names(self): # names such as 'expires' are not special in first name=value pair # of Set-Cookie: header - from cookielib import CookieJar - c = CookieJar() interact_netscape(c, "http://www.acme.com/", 'expires=eggs') interact_netscape(c, "http://www.acme.com/", 'version=eggs; spam=eggs') @@ -469,8 +446,6 @@ self.assert_('version' in cookies) def test_expires(self): - from cookielib import time2netscape, CookieJar - # if expires is in future, keep cookie... c = CookieJar() future = time2netscape(time.time()+3600) @@ -509,8 +484,6 @@ # XXX RFC 2965 expiry rules (some apply to V0 too) def test_default_path(self): - from cookielib import CookieJar, DefaultCookiePolicy - # RFC 2965 pol = DefaultCookiePolicy(rfc2965=True) @@ -551,7 +524,6 @@ self.assert_("/blah/rhubarb" in c._cookies["www.acme.com"]) def test_escape_path(self): - from cookielib import escape_path cases = [ # quoted safe ("/foo%2f/bar", "/foo%2F/bar"), @@ -575,57 +547,50 @@ self.assertEquals(escape_path(arg), result) def test_request_path(self): - from urllib2 import Request - from cookielib import request_path # with parameters - req = Request("http://www.example.com/rheum/rhaponicum;" - "foo=bar;sing=song?apples=pears&spam=eggs#ni") + req = urllib2.Request("http://www.example.com/rheum/rhaponicum;" + "foo=bar;sing=song?apples=pears&spam=eggs#ni") self.assertEquals(request_path(req), "/rheum/rhaponicum;" "foo=bar;sing=song?apples=pears&spam=eggs#ni") # without parameters - req = Request("http://www.example.com/rheum/rhaponicum?" - "apples=pears&spam=eggs#ni") + req = urllib2.Request("http://www.example.com/rheum/rhaponicum?" + "apples=pears&spam=eggs#ni") self.assertEquals(request_path(req), "/rheum/rhaponicum?" "apples=pears&spam=eggs#ni") # missing final slash - req = Request("http://www.example.com") + req = urllib2.Request("http://www.example.com") self.assertEquals(request_path(req), "/") def test_request_port(self): - from urllib2 import Request - from cookielib import request_port, DEFAULT_HTTP_PORT - req = Request("http://www.acme.com:1234/", - headers={"Host": "www.acme.com:4321"}) + req = urllib2.Request("http://www.acme.com:1234/", + headers={"Host": "www.acme.com:4321"}) self.assertEquals(request_port(req), "1234") - req = Request("http://www.acme.com/", - headers={"Host": "www.acme.com:4321"}) + req = urllib2.Request("http://www.acme.com/", + headers={"Host": "www.acme.com:4321"}) self.assertEquals(request_port(req), DEFAULT_HTTP_PORT) def test_request_host(self): - from urllib2 import Request - from cookielib import request_host # this request is illegal (RFC2616, 14.2.3) - req = Request("http://1.1.1.1/", - headers={"Host": "www.acme.com:80"}) + req = urllib2.Request("http://1.1.1.1/", + headers={"Host": "www.acme.com:80"}) # libwww-perl wants this response, but that seems wrong (RFC 2616, # section 5.2, point 1., and RFC 2965 section 1, paragraph 3) #self.assertEquals(request_host(req), "www.acme.com") self.assertEquals(request_host(req), "1.1.1.1") - req = Request("http://www.acme.com/", - headers={"Host": "irrelevant.com"}) + req = urllib2.Request("http://www.acme.com/", + headers={"Host": "irrelevant.com"}) self.assertEquals(request_host(req), "www.acme.com") # not actually sure this one is valid Request object, so maybe should # remove test for no host in url in request_host function? - req = Request("/resource.html", - headers={"Host": "www.acme.com"}) + req = urllib2.Request("/resource.html", + headers={"Host": "www.acme.com"}) self.assertEquals(request_host(req), "www.acme.com") # port shouldn't be in request-host - req = Request("http://www.acme.com:2345/resource.html", - headers={"Host": "www.acme.com:5432"}) + req = urllib2.Request("http://www.acme.com:2345/resource.html", + headers={"Host": "www.acme.com:5432"}) self.assertEquals(request_host(req), "www.acme.com") def test_is_HDN(self): - from cookielib import is_HDN self.assert_(is_HDN("foo.bar.com")) self.assert_(is_HDN("1foo2.3bar4.5com")) self.assert_(not is_HDN("192.168.1.1")) @@ -636,7 +601,6 @@ self.assert_(not is_HDN("foo.")) def test_reach(self): - from cookielib import reach self.assertEquals(reach("www.acme.com"), ".acme.com") self.assertEquals(reach("acme.com"), "acme.com") self.assertEquals(reach("acme.local"), ".local") @@ -647,7 +611,6 @@ self.assertEquals(reach("192.168.0.1"), "192.168.0.1") def test_domain_match(self): - from cookielib import domain_match, user_domain_match self.assert_(domain_match("192.168.1.1", "192.168.1.1")) self.assert_(not domain_match("192.168.1.1", ".168.1.1")) self.assert_(domain_match("x.y.com", "x.Y.com")) @@ -688,7 +651,6 @@ # domain are rejected. # XXX far from complete - from cookielib import CookieJar c = CookieJar() interact_2965(c, "http://www.nasty.com/", 'foo=bar; domain=friendly.org; Version="1"') @@ -697,8 +659,6 @@ def test_strict_domain(self): # Cookies whose domain is a country-code tld like .co.uk should # not be set if CookiePolicy.strict_domain is true. - from cookielib import CookieJar, DefaultCookiePolicy - cp = DefaultCookiePolicy(strict_domain=True) cj = CookieJar(policy=cp) interact_netscape(cj, "http://example.co.uk/", 'no=problemo') @@ -714,8 +674,6 @@ # Netscape: .www.bar.com, www.bar.com, .bar.com, bar.com, no domain # should all get accepted, as should .acme.com, acme.com and no domain # for 2-component domains like acme.com. - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar() # two-component V0 domain is OK @@ -761,8 +719,6 @@ self.assertEquals(len(c), 4) def test_two_component_domain_rfc2965(self): - from cookielib import CookieJar, DefaultCookiePolicy - pol = DefaultCookiePolicy(rfc2965=True) c = CookieJar(pol) @@ -807,31 +763,28 @@ self.assertEquals(len(c), 3) def test_domain_allow(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - c = CookieJar(policy=DefaultCookiePolicy( blocked_domains=["acme.com"], allowed_domains=["www.acme.com"])) - req = Request("http://acme.com/") + req = urllib2.Request("http://acme.com/") headers = ["Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/"] res = FakeResponse(headers, "http://acme.com/") c.extract_cookies(res, req) self.assertEquals(len(c), 0) - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) self.assertEquals(len(c), 1) - req = Request("http://www.coyote.com/") + req = urllib2.Request("http://www.coyote.com/") res = FakeResponse(headers, "http://www.coyote.com/") c.extract_cookies(res, req) self.assertEquals(len(c), 1) # set a cookie with non-allowed domain... - req = Request("http://www.coyote.com/") + req = urllib2.Request("http://www.coyote.com/") res = FakeResponse(headers, "http://www.coyote.com/") cookies = c.make_cookies(res, req) c.set_cookie(cookies[0]) @@ -841,15 +794,12 @@ self.assert_(not req.has_header("Cookie")) def test_domain_block(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - pol = DefaultCookiePolicy( rfc2965=True, blocked_domains=[".acme.com"]) c = CookieJar(policy=pol) headers = ["Set-Cookie: CUSTOMER=WILE_E_COYOTE; path=/"] - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) self.assertEquals(len(c), 0) @@ -859,11 +809,11 @@ self.assertEquals(len(c), 1) c.clear() - req = Request("http://www.roadrunner.net/") + req = urllib2.Request("http://www.roadrunner.net/") res = FakeResponse(headers, "http://www.roadrunner.net/") c.extract_cookies(res, req) self.assertEquals(len(c), 1) - req = Request("http://www.roadrunner.net/") + req = urllib2.Request("http://www.roadrunner.net/") c.add_cookie_header(req) self.assert_((req.has_header("Cookie") and req.has_header("Cookie2"))) @@ -874,7 +824,7 @@ self.assertEquals(len(c), 1) # set a cookie with blocked domain... - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") res = FakeResponse(headers, "http://www.acme.com/") cookies = c.make_cookies(res, req) c.set_cookie(cookies[0]) @@ -884,8 +834,6 @@ self.assert_(not req.has_header("Cookie")) def test_secure(self): - from cookielib import CookieJar, DefaultCookiePolicy - for ns in True, False: for whitespace in " ", "": c = CookieJar() @@ -909,7 +857,6 @@ "secure cookie registered non-secure") def test_quote_cookie_value(self): - from cookielib import CookieJar, DefaultCookiePolicy c = CookieJar(policy=DefaultCookiePolicy(rfc2965=True)) interact_2965(c, "http://www.acme.com/", r'foo=\b"a"r; Version=1') h = interact_2965(c, "http://www.acme.com/") @@ -917,19 +864,15 @@ def test_missing_final_slash(self): # Missing slash from request URL's abs_path should be assumed present. - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request url = "http://www.acme.com" c = CookieJar(DefaultCookiePolicy(rfc2965=True)) interact_2965(c, url, "foo=bar; Version=1") - req = Request(url) + req = urllib2.Request(url) self.assertEquals(len(c), 1) c.add_cookie_header(req) self.assert_(req.has_header("Cookie")) def test_domain_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - pol = DefaultCookiePolicy(rfc2965=True) c = CookieJar(pol) @@ -953,8 +896,6 @@ self.assert_('$Domain="bar.com"' in h, "domain not returned") def test_path_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - pol = DefaultCookiePolicy(rfc2965=True) c = CookieJar(pol) @@ -971,8 +912,6 @@ self.assert_('$Path="/"' in h, "path not returned") def test_port_mirror(self): - from cookielib import CookieJar, DefaultCookiePolicy - pol = DefaultCookiePolicy(rfc2965=True) c = CookieJar(pol) @@ -1005,8 +944,6 @@ "values") def test_no_return_comment(self): - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) url = "http://foo.bar.com/" interact_2965(c, url, 'spam=eggs; Version=1; ' @@ -1018,8 +955,6 @@ "Comment or CommentURL cookie-attributes returned to server") def test_Cookie_iterator(self): - from cookielib import CookieJar, Cookie, DefaultCookiePolicy - cs = CookieJar(DefaultCookiePolicy(rfc2965=True)) # add some random cookies interact_2965(cs, "http://blah.spam.org/", 'foo=eggs; Version=1; ' @@ -1052,8 +987,6 @@ i = i + 1 def test_parse_ns_headers(self): - from cookielib import parse_ns_headers - # missing domain value (invalid cookie) self.assertEquals( parse_ns_headers(["foo=bar; path=/; domain"]), @@ -1076,10 +1009,8 @@ def test_bad_cookie_header(self): def cookiejar_from_cookie_headers(headers): - from cookielib import CookieJar - from urllib2 import Request c = CookieJar() - req = Request("http://www.example.com/") + req = urllib2.Request("http://www.example.com/") r = FakeResponse(headers, "http://www.example.com/") c.extract_cookies(r, req) return c @@ -1108,9 +1039,6 @@ # Tests taken from libwww-perl, with a few modifications and additions. def test_netscape_example_1(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - #------------------------------------------------------------------- # First we check that it works for the original example at # http://www.netscape.com/newsref/std/cookie_spec.html @@ -1153,9 +1081,9 @@ c = CookieJar(DefaultCookiePolicy(rfc2965 = True)) - #req = Request("http://1.1.1.1/", + #req = urllib2.Request("http://1.1.1.1/", # headers={"Host": "www.acme.com:80"}) - req = Request("http://www.acme.com:80/", + req = urllib2.Request("http://www.acme.com:80/", headers={"Host": "www.acme.com:80"}) headers.append( @@ -1164,7 +1092,7 @@ res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") c.add_cookie_header(req) self.assertEqual(req.get_header("Cookie"), "CUSTOMER=WILE_E_COYOTE") @@ -1174,7 +1102,7 @@ res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) - req = Request("http://www.acme.com/foo/bar") + req = urllib2.Request("http://www.acme.com/foo/bar") c.add_cookie_header(req) h = req.get_header("Cookie") @@ -1185,7 +1113,7 @@ res = FakeResponse(headers, "http://www.acme.com") c.extract_cookies(res, req) - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") c.add_cookie_header(req) h = req.get_header("Cookie") @@ -1193,7 +1121,7 @@ "CUSTOMER=WILE_E_COYOTE" in h and "SHIPPING=FEDEX" not in h) - req = Request("http://www.acme.com/foo/") + req = urllib2.Request("http://www.acme.com/foo/") c.add_cookie_header(req) h = req.get_header("Cookie") @@ -1202,9 +1130,6 @@ h.startswith("SHIPPING=FEDEX;"))) def test_netscape_example_2(self): - from cookielib import CookieJar - from urllib2 import Request - # Second Example transaction sequence: # # Assume all mappings from above have been cleared. @@ -1231,13 +1156,13 @@ c = CookieJar() headers = [] - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") headers.append("Set-Cookie: PART_NUMBER=ROCKET_LAUNCHER_0001; path=/") res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) - req = Request("http://www.acme.com/") + req = urllib2.Request("http://www.acme.com/") c.add_cookie_header(req) self.assertEquals(req.get_header("Cookie"), @@ -1248,7 +1173,7 @@ res = FakeResponse(headers, "http://www.acme.com/") c.extract_cookies(res, req) - req = Request("http://www.acme.com/ammo") + req = urllib2.Request("http://www.acme.com/ammo") c.add_cookie_header(req) self.assert_(re.search(r"PART_NUMBER=RIDING_ROCKET_0023;\s*" @@ -1256,7 +1181,6 @@ req.get_header("Cookie"))) def test_ietf_example_1(self): - from cookielib import CookieJar, DefaultCookiePolicy #------------------------------------------------------------------- # Then we test with the examples from draft-ietf-http-state-man-mec-03.txt # @@ -1371,8 +1295,6 @@ # contains all the cookies received so far. def test_ietf_example_2(self): - from cookielib import CookieJar, DefaultCookiePolicy - # 5.2 Example 2 # # This example illustrates the effect of the Path attribute. All detail @@ -1428,8 +1350,6 @@ def test_rejection(self): # Test rejection of Set-Cookie2 responses based on domain, path, port. - from cookielib import DefaultCookiePolicy, LWPCookieJar - pol = DefaultCookiePolicy(rfc2965=True) c = LWPCookieJar(policy=pol) @@ -1522,8 +1442,6 @@ def test_url_encoding(self): # Try some URL encodings of the PATHs. # (the behaviour here has changed from libwww-perl) - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) interact_2965(c, "http://www.acme.com/foo%2f%25/%3c%3c%0Anew%E5/%E5", "foo = bar; version = 1") @@ -1543,8 +1461,6 @@ def test_mozilla(self): # Save / load Mozilla/Netscape cookie file format. - from cookielib import MozillaCookieJar, DefaultCookiePolicy - year_plus_one = time.localtime()[0] + 1 filename = support.TESTFN @@ -1586,12 +1502,9 @@ def test_netscape_misc(self): # Some additional Netscape cookies tests. - from cookielib import CookieJar - from urllib2 import Request - c = CookieJar() headers = [] - req = Request("http://foo.bar.acme.com/foo") + req = urllib2.Request("http://foo.bar.acme.com/foo") # Netscape allows a host part that contains dots headers.append("Set-Cookie: Customer=WILE_E_COYOTE; domain=.acme.com") @@ -1605,7 +1518,7 @@ res = FakeResponse(headers, "http://www.acme.com/foo") c.extract_cookies(res, req) - req = Request("http://foo.bar.acme.com/foo") + req = urllib2.Request("http://foo.bar.acme.com/foo") c.add_cookie_header(req) self.assert_( "PART_NUMBER=3,4" in req.get_header("Cookie") and @@ -1613,8 +1526,6 @@ def test_intranet_domains_2965(self): # Test handling of local intranet hostnames without a dot. - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar(DefaultCookiePolicy(rfc2965=True)) interact_2965(c, "http://example/", "foo1=bar; PORT; Discard; Version=1;") @@ -1627,8 +1538,6 @@ self.assert_("foo2=bar" in cookie and len(c) == 3) def test_intranet_domains_ns(self): - from cookielib import CookieJar, DefaultCookiePolicy - c = CookieJar(DefaultCookiePolicy(rfc2965 = False)) interact_netscape(c, "http://example/", "foo1=bar") cookie = interact_netscape(c, "http://example/", @@ -1641,9 +1550,6 @@ self.assertEquals(len(c), 2) def test_empty_path(self): - from cookielib import CookieJar, DefaultCookiePolicy - from urllib2 import Request - # Test for empty path # Broken web-server ORION/1.3.38 returns to the client response like # @@ -1654,12 +1560,12 @@ c = CookieJar(DefaultCookiePolicy(rfc2965 = True)) headers = [] - req = Request("http://www.ants.com/") + req = urllib2.Request("http://www.ants.com/") headers.append("Set-Cookie: JSESSIONID=ABCDERANDOM123; Path=") res = FakeResponse(headers, "http://www.ants.com/") c.extract_cookies(res, req) - req = Request("http://www.ants.com/") + req = urllib2.Request("http://www.ants.com/") c.add_cookie_header(req) self.assertEquals(req.get_header("Cookie"), @@ -1667,7 +1573,7 @@ self.assertEquals(req.get_header("Cookie2"), '$Version="1"') # missing path in the request URI - req = Request("http://www.ants.com:8080") + req = urllib2.Request("http://www.ants.com:8080") c.add_cookie_header(req) self.assertEquals(req.get_header("Cookie"), @@ -1675,15 +1581,12 @@ self.assertEquals(req.get_header("Cookie2"), '$Version="1"') def test_session_cookies(self): - from cookielib import CookieJar - from urllib2 import Request - year_plus_one = time.localtime()[0] + 1 # Check session cookies are deleted properly by # CookieJar.clear_session_cookies method - req = Request('http://www.perlmeister.com/scripts') + req = urllib2.Request('http://www.perlmeister.com/scripts') headers = [] headers.append("Set-Cookie: s1=session;Path=/scripts") headers.append("Set-Cookie: p1=perm; Domain=.perlmeister.com;" Copied: python/branches/py3k/Lib/test/test_http_cookies.py (from r63693, /python/branches/py3k/Lib/test/test_cookie.py) ============================================================================== --- /python/branches/py3k/Lib/test/test_cookie.py (original) +++ python/branches/py3k/Lib/test/test_http_cookies.py Mon May 26 18:32:26 2008 @@ -1,8 +1,8 @@ -# Simple test suite for Cookie.py +# Simple test suite for http/cookies.py from test.support import run_unittest, run_doctest import unittest -import Cookie +from http import cookies import warnings warnings.filterwarnings("ignore", @@ -34,7 +34,7 @@ ] for case in cases: - C = Cookie.SimpleCookie() + C = cookies.SimpleCookie() C.load(case['data']) self.assertEqual(repr(C), case['repr']) self.assertEqual(C.output(sep='\n'), case['output']) @@ -42,7 +42,7 @@ self.assertEqual(C[k].value, v) def test_load(self): - C = Cookie.SimpleCookie() + C = cookies.SimpleCookie() C.load('Customer="WILE_E_COYOTE"; Version=1; Path=/acme') self.assertEqual(C['Customer'].value, 'WILE_E_COYOTE') @@ -68,7 +68,7 @@ def test_quoted_meta(self): # Try cookie with quoted meta-data - C = Cookie.SimpleCookie() + C = cookies.SimpleCookie() C.load('Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"') self.assertEqual(C['Customer'].value, 'WILE_E_COYOTE') self.assertEqual(C['Customer']['version'], '1') @@ -76,7 +76,7 @@ def test_main(): run_unittest(CookieTests) - run_doctest(Cookie) + run_doctest(cookies) if __name__ == '__main__': test_main() Modified: python/branches/py3k/Lib/test/test_httplib.py ============================================================================== --- python/branches/py3k/Lib/test/test_httplib.py (original) +++ python/branches/py3k/Lib/test/test_httplib.py Mon May 26 18:32:26 2008 @@ -1,4 +1,4 @@ -import httplib +import http.client as httplib import io import socket @@ -48,8 +48,6 @@ # Some headers are added automatically, but should not be added by # .request() if they are explicitly set. - import httplib - class HeaderCountingBuffer(list): def __init__(self): self.count = {} Modified: python/branches/py3k/Lib/test/test_httpservers.py ============================================================================== --- python/branches/py3k/Lib/test/test_httpservers.py (original) +++ python/branches/py3k/Lib/test/test_httpservers.py Mon May 26 18:32:26 2008 @@ -4,16 +4,15 @@ Josip Dzolonga, and Michael Otteneder for the 2007/08 GHOP contest. """ -from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer -from SimpleHTTPServer import SimpleHTTPRequestHandler -from CGIHTTPServer import CGIHTTPRequestHandler +from http.server import BaseHTTPRequestHandler, HTTPServer, \ + SimpleHTTPRequestHandler, CGIHTTPRequestHandler import os import sys import base64 import shutil import urllib -import httplib +import http.client import tempfile import threading @@ -59,7 +58,7 @@ self.thread.stop() def request(self, uri, method='GET', body=None, headers={}): - self.connection = httplib.HTTPConnection('localhost', self.PORT) + self.connection = http.client.HTTPConnection('localhost', self.PORT) self.connection.request(method, uri, body, headers) return self.connection.getresponse() @@ -92,7 +91,7 @@ def setUp(self): BaseTestCase.setUp(self) - self.con = httplib.HTTPConnection('localhost', self.PORT) + self.con = http.client.HTTPConnection('localhost', self.PORT) self.con.connect() def test_command(self): @@ -343,7 +342,7 @@ def test_main(verbose=None): try: cwd = os.getcwd() - support.run_unittest(#BaseHTTPServerTestCase, + support.run_unittest(BaseHTTPServerTestCase, SimpleHTTPServerTestCase, CGIHTTPServerTestCase ) Modified: python/branches/py3k/Lib/test/test_pyclbr.py ============================================================================== --- python/branches/py3k/Lib/test/test_pyclbr.py (original) +++ python/branches/py3k/Lib/test/test_pyclbr.py Mon May 26 18:32:26 2008 @@ -168,7 +168,6 @@ 'getproxies_internetconfig',)) # not on all platforms cm('pickle') cm('aifc', ignore=('openfp',)) # set with = in module - cm('Cookie', ignore=('Cookie',)) # Cookie is an alias for SmartCookie cm('sre_parse', ignore=('dump',)) # from sre_constants import * cm('pdb') cm('pydoc') Modified: python/branches/py3k/Lib/test/test_shelve.py ============================================================================== --- python/branches/py3k/Lib/test/test_shelve.py (original) +++ python/branches/py3k/Lib/test/test_shelve.py Mon May 26 18:32:26 2008 @@ -3,7 +3,7 @@ import glob from test import support from collections import MutableMapping -from test.test_anydbm import dbm_iterator +from test.test_dbm import dbm_iterator def L1(s): return s.decode("latin-1") Modified: python/branches/py3k/Lib/test/test_socket.py ============================================================================== --- python/branches/py3k/Lib/test/test_socket.py (original) +++ python/branches/py3k/Lib/test/test_socket.py Mon May 26 18:32:26 2008 @@ -855,7 +855,7 @@ In this case (and in this case only), it should be possible to create a file object, read a line from it, create another file object, read another line from it, without loss of data in the - first file object's buffer. Note that httplib relies on this + first file object's buffer. Note that http.client relies on this when reading multiple requests from the same socket.""" bufsize = 0 # Use unbuffered mode Modified: python/branches/py3k/Lib/test/test_ssl.py ============================================================================== --- python/branches/py3k/Lib/test/test_ssl.py (original) +++ python/branches/py3k/Lib/test/test_ssl.py Mon May 26 18:32:26 2008 @@ -15,8 +15,7 @@ import traceback import asyncore -from BaseHTTPServer import HTTPServer -from SimpleHTTPServer import SimpleHTTPRequestHandler +from http.server import HTTPServer, SimpleHTTPRequestHandler # Optionally test SSL support, if we have it in the tested platform skip_expected = False Modified: python/branches/py3k/Lib/test/test_sundry.py ============================================================================== --- python/branches/py3k/Lib/test/test_sundry.py (original) +++ python/branches/py3k/Lib/test/test_sundry.py Mon May 26 18:32:26 2008 @@ -8,7 +8,6 @@ class TestUntestedModules(unittest.TestCase): def test_at_least_import_untested_modules(self): with support.catch_warning(): - import CGIHTTPServer import aifc import bdb import cgitb Modified: python/branches/py3k/Lib/test/test_urllib.py ============================================================================== --- python/branches/py3k/Lib/test/test_urllib.py (original) +++ python/branches/py3k/Lib/test/test_urllib.py Mon May 26 18:32:26 2008 @@ -1,7 +1,7 @@ """Regresssion tests for urllib""" import urllib -import httplib +import http.client import io import unittest from test import support @@ -107,14 +107,14 @@ def readline(self, length=None): if self.closed: return b"" return io.BytesIO.readline(self, length) - class FakeHTTPConnection(httplib.HTTPConnection): + class FakeHTTPConnection(http.client.HTTPConnection): def connect(self): self.sock = FakeSocket(fakedata) - self._connection_class = httplib.HTTPConnection - httplib.HTTPConnection = FakeHTTPConnection + self._connection_class = http.client.HTTPConnection + http.client.HTTPConnection = FakeHTTPConnection def unfakehttp(self): - httplib.HTTPConnection = self._connection_class + http.client.HTTPConnection = self._connection_class def test_read(self): self.fakehttp(b"Hello!") Modified: python/branches/py3k/Lib/test/test_urllib2.py ============================================================================== --- python/branches/py3k/Lib/test/test_urllib2.py (original) +++ python/branches/py3k/Lib/test/test_urllib2.py Mon May 26 18:32:26 2008 @@ -77,7 +77,7 @@ Note the case normalization of header names here, to .capitalize()-case. This should be preserved for backwards-compatibility. (In the HTTP case, normalization to .title()-case is done by urllib2 before sending headers to - httplib). + http.client). >>> url = "http://example.com" >>> r = Request(url, headers={"Spam-eggs": "blah"}) @@ -348,12 +348,12 @@ self._count = 0 self.requests = [] def http_open(self, req): - import mimetools, httplib, copy + import mimetools, http.client, copy from io import StringIO self.requests.append(copy.deepcopy(req)) if self._count == 0: self._count = self._count + 1 - name = httplib.responses[self.code] + name = http.client.responses[self.code] msg = mimetools.Message(StringIO(self.headers)) return self.parent.error( "http", req, MockFile(), self.code, name, msg) @@ -875,9 +875,8 @@ def test_cookie_redirect(self): # cookies shouldn't leak into redirected requests - from cookielib import CookieJar - - from test.test_cookielib import interact_netscape + from http.cookiejar import CookieJar + from test.test_http_cookiejar import interact_netscape cj = CookieJar() interact_netscape(cj, "http://www.example.com/", "spam=eggs") Modified: python/branches/py3k/Lib/test/test_urllib2_localnet.py ============================================================================== --- python/branches/py3k/Lib/test/test_urllib2_localnet.py (original) +++ python/branches/py3k/Lib/test/test_urllib2_localnet.py Mon May 26 18:32:26 2008 @@ -4,29 +4,29 @@ import threading import urlparse import urllib2 -import BaseHTTPServer +import http.server import unittest import hashlib from test import support # Loopback http server infrastructure -class LoopbackHttpServer(BaseHTTPServer.HTTPServer): +class LoopbackHttpServer(http.server.HTTPServer): """HTTP server w/ a few modifications that make it useful for loopback testing purposes. """ def __init__(self, server_address, RequestHandlerClass): - BaseHTTPServer.HTTPServer.__init__(self, - server_address, - RequestHandlerClass) + http.server.HTTPServer.__init__(self, + server_address, + RequestHandlerClass) # Set the timeout of our listening socket really low so # that we can stop the server easily. self.socket.settimeout(1.0) def get_request(self): - """BaseHTTPServer method, overridden.""" + """HTTPServer method, overridden.""" request, client_address = self.socket.accept() @@ -188,7 +188,7 @@ # Proxy test infrastructure -class FakeProxyHandler(BaseHTTPServer.BaseHTTPRequestHandler): +class FakeProxyHandler(http.server.BaseHTTPRequestHandler): """This is a 'fake proxy' that makes it look like the entire internet has gone down due to a sudden zombie invasion. It main utility is in providing us with authentication support for @@ -283,7 +283,7 @@ def GetRequestHandler(responses): - class FakeHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): + class FakeHTTPRequestHandler(http.server.BaseHTTPRequestHandler): server_version = "TestHTTP/" requests = [] Modified: python/branches/py3k/Lib/test/test_urllib2net.py ============================================================================== --- python/branches/py3k/Lib/test/test_urllib2net.py (original) +++ python/branches/py3k/Lib/test/test_urllib2net.py Mon May 26 18:32:26 2008 @@ -33,7 +33,7 @@ ## could be used to HTTP authentication. # # def test_basic_auth(self): -# import httplib +# import http.client # # test_url = "http://www.python.org/test/test_urllib2/basic_auth" # test_hostport = "www.python.org" @@ -61,14 +61,14 @@ # # reasons, let's not implement it! (it's already implemented for proxy # # specification strings (that is, URLs or authorities specifying a # # proxy), so we must keep that) -# self.assertRaises(httplib.InvalidURL, +# self.assertRaises(http.client.InvalidURL, # urllib2.urlopen, "http://evil:thing at example.com") class CloseSocketTest(unittest.TestCase): def test_close(self): - import socket, httplib, gc + import socket, http.client, gc # calling .close() on urllib2's response objects should close the # underlying socket @@ -77,7 +77,7 @@ response = _urlopen_with_retry("http://www.python.org/") abused_fileobject = response.fp httpresponse = abused_fileobject.raw - self.assert_(httpresponse.__class__ is httplib.HTTPResponse) + self.assert_(httpresponse.__class__ is http.client.HTTPResponse) fileobject = httpresponse.fp self.assert_(not fileobject.closed) Modified: python/branches/py3k/Lib/test/test_xmlrpc.py ============================================================================== --- python/branches/py3k/Lib/test/test_xmlrpc.py (original) +++ python/branches/py3k/Lib/test/test_xmlrpc.py Mon May 26 18:32:26 2008 @@ -7,7 +7,7 @@ import xmlrpc.server import threading import mimetools -import httplib +import http.client import socket import os from test import support @@ -340,9 +340,9 @@ # [ch] The test 404 is causing lots of false alarms. def XXXtest_404(self): - # send POST with httplib, it should return 404 header and + # send POST with http.client, it should return 404 header and # 'Not Found' message. - conn = httplib.HTTPConnection('localhost', PORT) + conn = http.client.HTTPConnection('localhost', PORT) conn.request('POST', '/this-is-not-valid') response = conn.getresponse() conn.close() Modified: python/branches/py3k/Lib/test/test_xmlrpc_net.py ============================================================================== --- python/branches/py3k/Lib/test/test_xmlrpc_net.py (original) +++ python/branches/py3k/Lib/test/test_xmlrpc_net.py Mon May 26 18:32:26 2008 @@ -6,7 +6,7 @@ import unittest from test import support -import xmlrpclib.client as xmlrpclib +import xmlrpc.client as xmlrpclib class CurrentTimeTest(unittest.TestCase): Modified: python/branches/py3k/Lib/urllib.py ============================================================================== --- python/branches/py3k/Lib/urllib.py (original) +++ python/branches/py3k/Lib/urllib.py Mon May 26 18:32:26 2008 @@ -22,7 +22,7 @@ (mimetools.Message objects are queried with the getheader() method.) """ -import httplib +import http.client import os import socket import sys @@ -352,7 +352,7 @@ try: response = http_conn.getresponse() - except httplib.BadStatusLine: + except http.client.BadStatusLine: # something went wrong with the HTTP status line raise IOError('http protocol error', 0, 'got a bad status line', None) @@ -369,7 +369,7 @@ def open_http(self, url, data=None): """Use HTTP protocol.""" - return self._open_generic_http(httplib.HTTPConnection, url, data) + return self._open_generic_http(http.client.HTTPConnection, url, data) def http_error(self, url, fp, errcode, errmsg, headers, data=None): """Handle http errors. @@ -395,9 +395,9 @@ if _have_ssl: def _https_connection(self, host): - return httplib.HTTPSConnection(host, - key_file=self.key_file, - cert_file=self.cert_file) + return http.client.HTTPSConnection(host, + key_file=self.key_file, + cert_file=self.cert_file) def open_https(self, url, data=None): """Use HTTPS protocol.""" Modified: python/branches/py3k/Lib/urllib2.py ============================================================================== --- python/branches/py3k/Lib/urllib2.py (original) +++ python/branches/py3k/Lib/urllib2.py Mon May 26 18:32:26 2008 @@ -89,7 +89,7 @@ import base64 import hashlib -import httplib +import http.client import io import mimetools import os @@ -441,7 +441,7 @@ default_classes = [ProxyHandler, UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, FTPHandler, FileHandler, HTTPErrorProcessor] - if hasattr(httplib, 'HTTPS'): + if hasattr(http.client, 'HTTPS'): default_classes.append(HTTPSHandler) skip = set() for klass in default_classes: @@ -1047,7 +1047,7 @@ def do_open(self, http_class, req): """Return an addinfourl object for the request, using http_class. - http_class must implement the HTTPConnection API from httplib. + http_class must implement the HTTPConnection API from http.client. The addinfourl return value is a file-like object. It also has methods and attributes including: - info(): return a mimetools.Message object for the headers @@ -1082,7 +1082,7 @@ # object initialized properly. # XXX Should an HTTPResponse object really be passed to - # BufferedReader? If so, we should change httplib to support + # BufferedReader? If so, we should change http.client to support # this use directly. # Add some fake methods to the reader to satisfy BufferedReader. @@ -1101,23 +1101,23 @@ class HTTPHandler(AbstractHTTPHandler): def http_open(self, req): - return self.do_open(httplib.HTTPConnection, req) + return self.do_open(http.client.HTTPConnection, req) http_request = AbstractHTTPHandler.do_request_ -if hasattr(httplib, 'HTTPS'): +if hasattr(http.client, 'HTTPS'): class HTTPSHandler(AbstractHTTPHandler): def https_open(self, req): - return self.do_open(httplib.HTTPSConnection, req) + return self.do_open(http.client.HTTPSConnection, req) https_request = AbstractHTTPHandler.do_request_ class HTTPCookieProcessor(BaseHandler): def __init__(self, cookiejar=None): - import cookielib + import http.cookiejar if cookiejar is None: - cookiejar = cookielib.CookieJar() + cookiejar = http.cookiejar.CookieJar() self.cookiejar = cookiejar def http_request(self, request): Modified: python/branches/py3k/Lib/wsgiref/simple_server.py ============================================================================== --- python/branches/py3k/Lib/wsgiref/simple_server.py (original) +++ python/branches/py3k/Lib/wsgiref/simple_server.py Mon May 26 18:32:26 2008 @@ -10,7 +10,7 @@ module. See also the BaseHTTPServer module docs for other API information. """ -from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +from http.server import BaseHTTPRequestHandler, HTTPServer import urllib, sys from wsgiref.handlers import SimpleHandler Modified: python/branches/py3k/Lib/xmlrpc/client.py ============================================================================== --- python/branches/py3k/Lib/xmlrpc/client.py (original) +++ python/branches/py3k/Lib/xmlrpc/client.py Mon May 26 18:32:26 2008 @@ -135,7 +135,7 @@ """ import re, time, operator -import httplib +import http.client # -------------------------------------------------------------------- # Internal stuff @@ -1196,7 +1196,7 @@ def send_request(self, host, handler, request_body, debug): host, extra_headers, x509 = self.get_host_info(host) - connection = httplib.HTTPConnection(host) + connection = http.client.HTTPConnection(host) if debug: connection.set_debuglevel(1) headers = {} @@ -1261,10 +1261,10 @@ import socket if not hasattr(socket, "ssl"): raise NotImplementedError( - "your version of httplib doesn't support HTTPS") + "your version of http.client doesn't support HTTPS") host, extra_headers, x509 = self.get_host_info(host) - connection = httplib.HTTPSConnection(host, None, **(x509 or {})) + connection = http.client.HTTPSConnection(host, None, **(x509 or {})) if debug: connection.set_debuglevel(1) headers = {} Modified: python/branches/py3k/Lib/xmlrpc/server.py ============================================================================== --- python/branches/py3k/Lib/xmlrpc/server.py (original) +++ python/branches/py3k/Lib/xmlrpc/server.py Mon May 26 18:32:26 2008 @@ -105,8 +105,9 @@ # Based on code written by Fredrik Lundh. from xmlrpc.client import Fault, dumps, loads +from http.server import BaseHTTPRequestHandler +import http.server import socketserver -import BaseHTTPServer import sys import os import re @@ -408,7 +409,7 @@ else: raise Exception('method "%s" is not supported' % method) -class SimpleXMLRPCRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): +class SimpleXMLRPCRequestHandler(BaseHTTPRequestHandler): """Simple XML-RPC request handler class. Handles all HTTP POST requests and attempts to decode them as @@ -500,7 +501,7 @@ """Selectively log an accepted request.""" if self.server.logRequests: - BaseHTTPServer.BaseHTTPRequestHandler.log_request(self, code, size) + BaseHTTPRequestHandler.log_request(self, code, size) class SimpleXMLRPCServer(socketserver.TCPServer, SimpleXMLRPCDispatcher): @@ -560,10 +561,9 @@ """ code = 400 - message, explain = \ - BaseHTTPServer.BaseHTTPRequestHandler.responses[code] + message, explain = BaseHTTPRequestHandler.responses[code] - response = BaseHTTPServer.DEFAULT_ERROR_MESSAGE % \ + response = http.server.DEFAULT_ERROR_MESSAGE % \ { 'code' : code, 'message' : message, Modified: python/branches/py3k/Misc/cheatsheet ============================================================================== --- python/branches/py3k/Misc/cheatsheet (original) +++ python/branches/py3k/Misc/cheatsheet Mon May 26 18:32:26 2008 @@ -1795,14 +1795,10 @@ Standard library modules Operation Result aifc Stuff to parse AIFF-C and AIFF files. -dbm Generic interface to all dbm clones. (dbm.bsd, dbm.gnu, - dbm.ndbm, dbm.dumb) asynchat Support for 'chat' style protocols asyncore Asynchronous File I/O (in select style) atexit Register functions to be called at exit of Python interpreter. base64 Conversions to/from base64 RFC-MIME transport encoding . -BaseHTTPServer Base class forhttp services. -Bastion "Bastionification" utility (control access to instance vars) bdb A generic Python debugger base class. binhex Macintosh binhex compression/decompression. bisect List bisection algorithms. @@ -1810,7 +1806,6 @@ calendar Calendar printing functions. cgi Wraps the WWW Forms Common Gateway Interface (CGI). cgitb Utility for handling CGI tracebacks. -CGIHTTPServer CGI http services. cmd A generic class to build line-oriented command interpreters. datetime Basic date and time types. code Utilities needed to emulate Python's interactive interpreter @@ -1818,10 +1813,12 @@ colorsys Conversion functions between RGB and other color systems. commands Tools for executing UNIX commands . compileall Force "compilation" of all .py files in a directory. -ConfigParser Configuration file parser (much like windows .ini files) +configparser Configuration file parser (much like windows .ini files) copy Generic shallow and deep copying operations. -copy_reg Helper to provide extensibility for pickle/cPickle. +copyreg Helper to provide extensibility for pickle/cPickle. csv Read and write files with comma separated values. +dbm Generic interface to all dbm clones (dbm.bsd, dbm.gnu, + dbm.ndbm, dbm.dumb). dircache Sorted list of files in a dir, using a cache. difflib Tool for creating delta between sequences. dis Bytecode disassembler. @@ -1844,11 +1841,11 @@ glob filename globbing. gzip Read & write gzipped files. heapq Priority queue implemented using lists organized as heaps. -HMAC Keyed-Hashing for Message Authentication -- RFC 2104. -htmlentitydefs Proposed entity definitions for HTML. -htmllib HTML parsing utilities. -HTMLParser A parser for HTML and XHTML. -httplib HTTP client class. +hmac Keyed-Hashing for Message Authentication -- RFC 2104. +html.entities HTML entity definitions. +html.parser A parser for HTML and XHTML. +http.client HTTP client class. +http.server HTTP server services. ihooks Hooks into the "import" mechanism. imaplib IMAP4 client.Based on RFC 2060. imghdr Recognizing image files based on their first few bytes. @@ -1864,7 +1861,6 @@ mailbox A class to handle a unix-style or mmdf-style mailbox. mailcap Mailcap file handling (RFC 1524). mhlib MH (mailbox) interface. -mimetools Various tools used by MIME-reading or MIME-writing programs. mimetypes Guess the MIME type of a file. mmap Interface to memory-mapped files - they behave like mutable strings./font> @@ -1892,13 +1888,11 @@ pyexpat Interface to the Expay XML parser. py_compile Routine to "compile" a .py file to a .pyc file. pyclbr Parse a Python file and retrieve classes and methods. -Queue A multi-producer, multi-consumer queue. +queue A multi-producer, multi-consumer queue. quopri Conversions to/from quoted-printable transport encoding. random Random variable generators re Regular Expressions. -repr Redo repr() but with limits on most sizes. -rexec Restricted execution facilities ("safe" exec, eval, etc). -rfc822 RFC-822 message manipulation class. +reprlib Redo repr() but with limits on most sizes. rlcompleter Word completion for GNU readline 2.0. robotparser Parse robots.txt files, useful for web spiders. sched A generally useful event scheduler class. @@ -1906,7 +1900,6 @@ shelve Manage shelves of pickled objects. shlex Lexical analyzer class for simple shell-like syntaxes. shutil Utility functions usable in a shell-like program. -SimpleHTTPServer Simple extension to base http class site Append module search paths for third-party packages to sys.path. smtplib SMTP Client class (RFC 821) @@ -1916,8 +1909,6 @@ statvfs Constants for interpreting statvfs struct as returned by os.statvfs()and os.fstatvfs() (if they exist). string A collection of string operations. -StringIO File-like objects that read/write a string buffer (a fasterC - implementation exists in built-in module: cStringIO). sunau Stuff to parse Sun and NeXT audio files. sunaudio Interpret sun audio headers. symbol Non-terminal symbols of Python grammar (from "graminit.h"). @@ -1927,7 +1918,6 @@ tempfile Temporary file name allocation. textwrap Object for wrapping and filling text. threading Proposed new higher-level threading interfaces -threading_api (doc of the threading module) token Tokens (from "token.h"). tokenize Compiles a regular expression that recognizes Python tokens. traceback Format and print Python stack traces. @@ -1939,17 +1929,13 @@ urllib Open an arbitrary URL. urlparse Parse URLs according to latest draft of standard. user Hook to allow user-specified customization code to run. -UserDict A wrapper to allow subclassing of built-in dict class. -UserList A wrapper to allow subclassing of built-in list class. -UserString A wrapper to allow subclassing of built-in string class. uu UUencode/UUdecode. unittest Utilities for implementing unit testing. wave Stuff to parse WAVE files. weakref Tools for creating and managing weakly referenced objects. webbrowser Platform independent URL launcher. xdrlib Implements (a subset of) Sun XDR (eXternal Data - Representation) -xmllib A parser for XML, using the derived class as static DTD. + Representation). xml.dom Classes for processing XML using the Document Object Model. xml.sax Classes for processing XML using the SAX API. xmlrpc.client Support for remote procedure calls using XML. From python-3000-checkins at python.org Mon May 26 19:02:12 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Mon, 26 May 2008 19:02:12 +0200 (CEST) Subject: [Python-3000-checkins] r63702 - in python/branches/py3k: Lib/distutils/config.py Message-ID: <20080526170212.711F21E4008@bag.python.org> Author: georg.brandl Date: Mon May 26 19:01:57 2008 New Revision: 63702 Log: Merged revisions 63575 via svnmerge from svn+ssh://pythondev at svn.python.org/python/trunk ........ r63575 | martin.v.loewis | 2008-05-24 11:00:04 +0200 (Sat, 24 May 2008) | 3 lines Use announce instead of print, to suppress output in the testsuite. ........ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Lib/distutils/config.py Modified: python/branches/py3k/Lib/distutils/config.py ============================================================================== --- python/branches/py3k/Lib/distutils/config.py (original) +++ python/branches/py3k/Lib/distutils/config.py Mon May 26 19:01:57 2008 @@ -58,7 +58,7 @@ """Reads the .pypirc file.""" rc = self._get_rc_file() if os.path.exists(rc): - print('Using PyPI login from %s' % rc) + self.announce('Using PyPI login from %s' % rc) repository = self.repository or self.DEFAULT_REPOSITORY realm = self.realm or self.DEFAULT_REALM From python-3000-checkins at python.org Mon May 26 19:36:48 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 19:36:48 +0200 (CEST) Subject: [Python-3000-checkins] r63703 - in python/branches/py3k: Doc/library/collections.rst Doc/library/struct.rst Lib/test/test_ioctl.py Lib/test/test_math.py Lib/test/test_minidom.py Lib/xml/dom/minidom.py Modules/mathmodule.c Message-ID: <20080526173648.6D0D51E4017@bag.python.org> Author: benjamin.peterson Date: Mon May 26 19:36:47 2008 New Revision: 63703 Log: Merged revisions 63542-63544,63546,63553,63563-63564,63567,63569,63576 via svnmerge from svn+ssh://pythondev at svn.python.org/python/trunk ........ r63542 | mark.dickinson | 2008-05-22 20:35:30 -0500 (Thu, 22 May 2008) | 5 lines Issue #2819: Add math.sum, a function that sums a sequence of floats efficiently but with no intermediate loss of precision. Based on Raymond Hettinger's ASPN recipe. Thanks Jean Brouwers for the patch. ........ r63543 | mark.dickinson | 2008-05-22 21:36:48 -0500 (Thu, 22 May 2008) | 2 lines Add tests for math.sum (Issue #2819) ........ r63544 | mark.dickinson | 2008-05-22 22:30:01 -0500 (Thu, 22 May 2008) | 2 lines Better error reporting in test_math.py ........ r63546 | raymond.hettinger | 2008-05-22 23:32:43 -0500 (Thu, 22 May 2008) | 1 line Tweak the comments and formatting. ........ r63553 | mark.dickinson | 2008-05-23 07:07:36 -0500 (Fri, 23 May 2008) | 3 lines Skip math.sum tests on non IEEE 754 platforms, and on IEEE 754 platforms that exhibit the problem described in issue #2937. ........ r63563 | martin.v.loewis | 2008-05-23 10:18:28 -0500 (Fri, 23 May 2008) | 3 lines Issue #1390: Raise ValueError in toxml when an invalid comment would otherwise be produced. ........ r63564 | raymond.hettinger | 2008-05-23 12:21:44 -0500 (Fri, 23 May 2008) | 1 line Issue 2909: show how to name unpacked fields. ........ r63567 | raymond.hettinger | 2008-05-23 12:34:34 -0500 (Fri, 23 May 2008) | 1 line Fix typo ........ r63569 | martin.v.loewis | 2008-05-23 14:33:13 -0500 (Fri, 23 May 2008) | 3 lines Mention that the leaking of variables from list comprehensions is fixed in 3.0. ........ r63576 | martin.v.loewis | 2008-05-24 04:36:45 -0500 (Sat, 24 May 2008) | 3 lines Don't try to get the window size if it was never set before. Fixes the test failure on Solaris. ........ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Doc/library/collections.rst python/branches/py3k/Doc/library/struct.rst python/branches/py3k/Lib/test/test_ioctl.py python/branches/py3k/Lib/test/test_math.py python/branches/py3k/Lib/test/test_minidom.py python/branches/py3k/Lib/xml/dom/minidom.py python/branches/py3k/Modules/mathmodule.c Modified: python/branches/py3k/Doc/library/collections.rst ============================================================================== --- python/branches/py3k/Doc/library/collections.rst (original) +++ python/branches/py3k/Doc/library/collections.rst Mon May 26 19:36:47 2008 @@ -113,7 +113,7 @@ Since some set operations create new sets, the default mixin methods need a way to create new instances from an iterable. The class constructor is assumed to have a signature in the form ``ClassName(iterable)``. - That assumption is factored-out to a single internal classmethod called + That assumption is factored-out to an internal classmethod called :meth:`_from_iterable` which calls ``cls(iterable)`` to produce a new set. If the :class:`Set` mixin is being used in a class with a different constructor signature, you will need to override :meth:`from_iterable` Modified: python/branches/py3k/Doc/library/struct.rst ============================================================================== --- python/branches/py3k/Doc/library/struct.rst (original) +++ python/branches/py3k/Doc/library/struct.rst Mon May 26 19:36:47 2008 @@ -216,6 +216,16 @@ native size and alignment are in effect; standard size and alignment does not enforce any alignment. +Unpacked fields can be named by assigning them to variables or by wrapping +the result in a named tuple:: + + >>> record = 'raymond \x32\x12\x08\x01\x08' + >>> name, serialnum, school, gradelevel = unpack('<10sHHb', record) + + >>> from collections import namedtuple + >>> Student = namedtuple('Student', 'name serialnum school gradelevel') + >>> Student._make(unpack('<10sHHb', s)) + Student(name='raymond ', serialnum=4658, school=264, gradelevel=8) .. seealso:: Modified: python/branches/py3k/Lib/test/test_ioctl.py ============================================================================== --- python/branches/py3k/Lib/test/test_ioctl.py (original) +++ python/branches/py3k/Lib/test/test_ioctl.py Mon May 26 19:36:47 2008 @@ -52,13 +52,10 @@ set_winsz_opcode_maybe_neg, = struct.unpack("i", struct.pack("I", termios.TIOCSWINSZ)) - # We're just testing that these calls do not raise exceptions. - saved_winsz = fcntl.ioctl(mfd, termios.TIOCGWINSZ, "\0"*8) our_winsz = struct.pack("HHHH",80,25,0,0) # test both with a positive and potentially negative ioctl code new_winsz = fcntl.ioctl(mfd, set_winsz_opcode_pos, our_winsz) new_winsz = fcntl.ioctl(mfd, set_winsz_opcode_maybe_neg, our_winsz) - fcntl.ioctl(mfd, set_winsz_opcode_maybe_neg, saved_winsz) finally: os.close(mfd) os.close(sfd) Modified: python/branches/py3k/Lib/test/test_math.py ============================================================================== --- python/branches/py3k/Lib/test/test_math.py (original) +++ python/branches/py3k/Lib/test/test_math.py Mon May 26 19:36:47 2008 @@ -626,6 +626,158 @@ self.assertRaises(ValueError, math.sqrt, NINF) self.assert_(math.isnan(math.sqrt(NAN))) + def testSum(self): + # math.sum relies on exact rounding for correct operation. + # There's a known problem with IA32 floating-point that causes + # inexact rounding in some situations, and will cause the + # math.sum tests below to fail; see issue #2937. On non IEEE + # 754 platforms, and on IEEE 754 platforms that exhibit the + # problem described in issue #2937, we simply skip the whole + # test. + + if not float.__getformat__("double").startswith("IEEE"): + return + + # on IEEE 754 compliant machines, both of the expressions + # below should round to 10000000000000002.0. + if 1e16+2.999 != 1e16+2.9999: + return + + # Python version of math.sum algorithm, for comparison + def msum(iterable): + """Full precision sum of values in iterable. Returns the value of + the sum, rounded to the nearest representable floating-point number + using the round-half-to-even rule. + + """ + # Stage 1: accumulate partials + partials = [] + for x in iterable: + i = 0 + for y in partials: + if abs(x) < abs(y): + x, y = y, x + hi = x + y + lo = y - (hi - x) + if lo: + partials[i] = lo + i += 1 + x = hi + partials[i:] = [x] if x else [] + + # Stage 2: sum partials + if not partials: + return 0.0 + + # sum from the top, stopping as soon as the sum is inexact. + total = partials.pop() + while partials: + x = partials.pop() + old_total, total = total, total + x + error = x - (total - old_total) + if error != 0.0: + # adjust for correct rounding if necessary + if partials and (partials[-1] > 0.0) == (error > 0.0) and \ + total + 2*error - total == 2*error: + total += 2*error + break + return total + + from sys import float_info + maxfloat = float_info.max + twopow = 2.**(float_info.max_exp - 1) + + test_values = [ + ([], 0.0), + ([0.0], 0.0), + ([1e100, 1.0, -1e100, 1e-100, 1e50, -1.0, -1e50], 1e-100), + ([1e308, 1e308, -1e308], OverflowError), + ([-1e308, 1e308, 1e308], 1e308), + ([1e308, -1e308, 1e308], 1e308), + ([2.0**1023, 2.0**1023, -2.0**1000], OverflowError), + ([twopow, twopow, twopow, twopow, -twopow, -twopow, -twopow], + OverflowError), + ([2.0**53, -0.5, -2.0**-54], 2.0**53-1.0), + ([2.0**53, 1.0, 2.0**-100], 2.0**53+2.0), + ([2.0**53+10.0, 1.0, 2.0**-100], 2.0**53+12.0), + + ([2.0**53-4.0, 0.5, 2.0**-54], 2.0**53-3.0), + ([2.0**1023-2.0**970, -1.0, 2.0**1023], OverflowError), + ([maxfloat, maxfloat*2.**-54], maxfloat), + ([maxfloat, maxfloat*2.**-53], OverflowError), + ([1./n for n in range(1, 1001)], 7.4854708605503451), + ([(-1.)**n/n for n in range(1, 1001)], -0.69264743055982025), + ([1.7**(i+1)-1.7**i for i in range(1000)] + [-1.7**1000], -1.0), + ([INF, -INF, NAN], ValueError), + ([NAN, INF, -INF], ValueError), + ([INF, NAN, INF], ValueError), + + ([INF, INF], OverflowError), + ([INF, -INF], ValueError), + ([-INF, 1e308, 1e308, -INF], OverflowError), + ([2.0**1023-2.0**970, 0.0, 2.0**1023], OverflowError), + ([2.0**1023-2.0**970, 1.0, 2.0**1023], OverflowError), + ([2.0**1023, 2.0**1023], OverflowError), + ([2.0**1023, 2.0**1023, -1.0], OverflowError), + ([twopow, twopow, twopow, twopow, -twopow, -twopow], + OverflowError), + ([twopow, twopow, twopow, twopow, -twopow, twopow], OverflowError), + ([-twopow, -twopow, -twopow, -twopow], OverflowError), + + ([2.**1023, 2.**1023, -2.**971], OverflowError), + ([2.**1023, 2.**1023, -2.**970], OverflowError), + ([-2.**970, 2.**1023, 2.**1023, -2.**-1074], OverflowError), + ([ 2.**1023, 2.**1023, -2.**970, 2.**-1074], OverflowError), + ([-2.**1023, 2.**971, -2.**1023], -maxfloat), + ([-2.**1023, -2.**1023, 2.**970], OverflowError), + ([-2.**1023, -2.**1023, 2.**970, 2.**-1074], OverflowError), + ([-2.**-1074, -2.**1023, -2.**1023, 2.**970], OverflowError), + ([2.**930, -2.**980, 2.**1023, 2.**1023, twopow, -twopow], + OverflowError), + ([2.**1023, 2.**1023, -1e307], OverflowError), + ([1e16, 1., 1e-16], 10000000000000002.0), + ([1e16-2., 1.-2.**53, -(1e16-2.), -(1.-2.**53)], 0.0), + ] + + for i, (vals, s) in enumerate(test_values): + if isinstance(s, type) and issubclass(s, Exception): + try: + m = math.sum(vals) + except s: + pass + else: + self.fail("test %d failed: got %r, expected %r " + "for math.sum(%.100r)" % + (i, m, s.__name__, vals)) + else: + try: + self.assertEqual(math.sum(vals), s) + except OverflowError: + self.fail("test %d failed: got OverflowError, expected %r " + "for math.sum(%.100r)" % (i, s, vals)) + except ValueError: + self.fail("test %d failed: got ValueError, expected %r " + "for math.sum(%.100r)" % (i, s, vals)) + + # compare with output of msum above, but only when + # result isn't an IEEE special or an exception + if not math.isinf(s) and not math.isnan(s): + self.assertEqual(msum(vals), s) + + from random import random, gauss, shuffle + for j in range(1000): + vals = [7, 1e100, -7, -1e100, -9e-20, 8e-20] * 10 + s = 0 + for i in range(200): + v = gauss(0, random()) ** 7 - s + s += v + vals.append(v) + shuffle(vals) + + s = msum(vals) + self.assertEqual(msum(vals), math.sum(vals)) + + def testTan(self): self.assertRaises(TypeError, math.tan) self.ftest('tan(0)', math.tan(0), 0) @@ -763,6 +915,10 @@ message = (("Unexpected ValueError: %s\n " + "in test %s:%s(%r)\n") % (exc.args[0], id, fn, ar)) self.fail(message) + except OverflowError: + message = ("Unexpected OverflowError in " + + "test %s:%s(%r)\n" % (id, fn, ar)) + self.fail(message) self.ftest("%s:%s(%r)" % (id, fn, ar), result, er) def test_main(): Modified: python/branches/py3k/Lib/test/test_minidom.py ============================================================================== --- python/branches/py3k/Lib/test/test_minidom.py (original) +++ python/branches/py3k/Lib/test/test_minidom.py Mon May 26 19:36:47 2008 @@ -1314,6 +1314,11 @@ for i in range(len(n1.childNodes)): stack.append((n1.childNodes[i], n2.childNodes[i])) + def testSerializeCommentNodeWithDoubleHyphen(self): + doc = create_doc_without_doctype() + doc.appendChild(doc.createComment("foo--bar")) + self.assertRaises(ValueError, doc.toxml) + def test_main(): run_unittest(MinidomTest) Modified: python/branches/py3k/Lib/xml/dom/minidom.py ============================================================================== --- python/branches/py3k/Lib/xml/dom/minidom.py (original) +++ python/branches/py3k/Lib/xml/dom/minidom.py Mon May 26 19:36:47 2008 @@ -1132,6 +1132,8 @@ self.data = self.nodeValue = data def writexml(self, writer, indent="", addindent="", newl=""): + if "--" in self.data: + raise ValueError("'--' is not allowed in a comment node") writer.write("%s%s" % (indent, self.data, newl)) Modified: python/branches/py3k/Modules/mathmodule.c ============================================================================== --- python/branches/py3k/Modules/mathmodule.c (original) +++ python/branches/py3k/Modules/mathmodule.c Mon May 26 19:36:47 2008 @@ -362,6 +362,199 @@ FUNC1(tanh, tanh, 0, "tanh(x)\n\nReturn the hyperbolic tangent of x.") +/* Precision summation function as msum() by Raymond Hettinger in + , + enhanced with the exact partials sum and roundoff from Mark + Dickinson's post at . + See those links for more details, proofs and other references. + + Note 1: IEEE 754R floating point semantics are assumed, + but the current implementation does not re-establish special + value semantics across iterations (i.e. handling -Inf + Inf). + + Note 2: No provision is made for intermediate overflow handling; + therefore, sum([1e+308, 1e-308, 1e+308]) returns result 1e+308 while + sum([1e+308, 1e+308, 1e-308]) raises an OverflowError due to the + overflow of the first partial sum. + + Note 3: Aggressively optimizing compilers can potentially eliminate the + residual values needed for accurate summation. For instance, the statements + "hi = x + y; lo = y - (hi - x);" could be mis-transformed to + "hi = x + y; lo = 0.0;" which defeats the computation of residuals. + + Note 4: A similar implementation is in Modules/cmathmodule.c. + Be sure to update both when making changes. + + Note 5: The signature of math.sum() differs from __builtin__.sum() + because the start argument doesn't make sense in the context of + accurate summation. Since the partials table is collapsed before + returning a result, sum(seq2, start=sum(seq1)) may not equal the + accurate result returned by sum(itertools.chain(seq1, seq2)). +*/ + +#define NUM_PARTIALS 32 /* initial partials array size, on stack */ + +/* Extend the partials array p[] by doubling its size. */ +static int /* non-zero on error */ +_sum_realloc(double **p_ptr, Py_ssize_t n, + double *ps, Py_ssize_t *m_ptr) +{ + void *v = NULL; + Py_ssize_t m = *m_ptr; + + m += m; /* double */ + if (n < m && m < (PY_SSIZE_T_MAX / sizeof(double))) { + double *p = *p_ptr; + if (p == ps) { + v = PyMem_Malloc(sizeof(double) * m); + if (v != NULL) + memcpy(v, ps, sizeof(double) * n); + } + else + v = PyMem_Realloc(p, sizeof(double) * m); + } + if (v == NULL) { /* size overflow or no memory */ + PyErr_SetString(PyExc_MemoryError, "math sum partials"); + return 1; + } + *p_ptr = (double*) v; + *m_ptr = m; + return 0; +} + +/* Full precision summation of a sequence of floats. + + def msum(iterable): + partials = [] # sorted, non-overlapping partial sums + for x in iterable: + i = 0 + for y in partials: + if abs(x) < abs(y): + x, y = y, x + hi = x + y + lo = y - (hi - x) + if lo: + partials[i] = lo + i += 1 + x = hi + partials[i:] = [x] + return sum_exact(partials) + + Rounded x+y stored in hi with the roundoff stored in lo. Together hi+lo + are exactly equal to x+y. The inner loop applies hi/lo summation to each + partial so that the list of partial sums remains exact. + + Sum_exact() adds the partial sums exactly and correctly rounds the final + result (using the round-half-to-even rule). The items in partials remain + non-zero, non-special, non-overlapping and strictly increasing in + magnitude, but possibly not all having the same sign. + + Depends on IEEE 754 arithmetic guarantees and half-even rounding. +*/ + +static PyObject* +math_sum(PyObject *self, PyObject *seq) +{ + PyObject *item, *iter, *sum = NULL; + Py_ssize_t i, j, n = 0, m = NUM_PARTIALS; + double x, y, hi, lo=0.0, ps[NUM_PARTIALS], *p = ps; + + iter = PyObject_GetIter(seq); + if (iter == NULL) + return NULL; + + PyFPE_START_PROTECT("sum", Py_DECREF(iter); return NULL) + + for(;;) { /* for x in iterable */ + assert(0 <= n && n <= m); + assert((m == NUM_PARTIALS && p == ps) || + (m > NUM_PARTIALS && p != NULL)); + + item = PyIter_Next(iter); + if (item == NULL) { + if (PyErr_Occurred()) + goto _sum_error; + break; + } + x = PyFloat_AsDouble(item); + Py_DECREF(item); + if (PyErr_Occurred()) + goto _sum_error; + + for (i = j = 0; j < n; j++) { /* for y in partials */ + y = p[j]; + hi = x + y; + lo = fabs(x) < fabs(y) + ? x - (hi - y) + : y - (hi - x); + if (lo != 0.0) + p[i++] = lo; + x = hi; + } + + n = i; /* ps[i:] = [x] */ + if (x != 0.0) { + /* If non-finite, reset partials, effectively + adding subsequent items without roundoff + and yielding correct non-finite results, + provided IEEE 754 rules are observed */ + if (! Py_IS_FINITE(x)) + n = 0; + else if (n >= m && _sum_realloc(&p, n, ps, &m)) + goto _sum_error; + p[n++] = x; + } + } + + if (n > 0) { + hi = p[--n]; + if (Py_IS_FINITE(hi)) { + /* sum_exact(ps, hi) from the top, stop when the sum becomes inexact. */ + while (n > 0) { + x = p[--n]; + y = hi; + hi = x + y; + assert(fabs(x) < fabs(y)); + lo = x - (hi - y); + if (lo != 0.0) + break; + } + /* Little dance to allow half-even rounding across multiple partials. + Needed so that sum([1e-16, 1, 1e16]) will round-up to two instead + of down to zero (the 1e16 makes the 1 slightly closer to two). */ + if (n > 0 && ((lo < 0.0 && p[n-1] < 0.0) || + (lo > 0.0 && p[n-1] > 0.0))) { + y = lo * 2.0; + x = hi + y; + if (y == (x - hi)) + hi = x; + } + } + else { /* raise corresponding error */ + errno = Py_IS_NAN(hi) ? EDOM : ERANGE; + if (is_error(hi)) + goto _sum_error; + } + } + else /* default */ + hi = 0.0; + sum = PyFloat_FromDouble(hi); + +_sum_error: + PyFPE_END_PROTECT(hi) + Py_DECREF(iter); + if (p != ps) + PyMem_Free(p); + return sum; +} + +#undef NUM_PARTIALS + +PyDoc_STRVAR(math_sum_doc, +"sum(iterable)\n\n\ +Return an accurate floating point sum of values in the iterable.\n\ +Assumes IEEE-754 floating point arithmetic."); + static PyObject * math_trunc(PyObject *self, PyObject *number) { @@ -833,6 +1026,7 @@ {"sin", math_sin, METH_O, math_sin_doc}, {"sinh", math_sinh, METH_O, math_sinh_doc}, {"sqrt", math_sqrt, METH_O, math_sqrt_doc}, + {"sum", math_sum, METH_O, math_sum_doc}, {"tan", math_tan, METH_O, math_tan_doc}, {"tanh", math_tanh, METH_O, math_tanh_doc}, {"trunc", math_trunc, METH_O, math_trunc_doc}, From python-3000-checkins at python.org Mon May 26 19:44:34 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 19:44:34 +0200 (CEST) Subject: [Python-3000-checkins] r63705 - in python/branches/py3k: Doc/library/test.rst Lib/test/README Lib/test/support.py Lib/test/test_float.py Message-ID: <20080526174434.275481E4008@bag.python.org> Author: benjamin.peterson Date: Mon May 26 19:44:33 2008 New Revision: 63705 Log: remove test.support.have_unicode Modified: python/branches/py3k/Doc/library/test.rst python/branches/py3k/Lib/test/README python/branches/py3k/Lib/test/support.py python/branches/py3k/Lib/test/test_float.py Modified: python/branches/py3k/Doc/library/test.rst ============================================================================== --- python/branches/py3k/Doc/library/test.rst (original) +++ python/branches/py3k/Doc/library/test.rst Mon May 26 19:44:33 2008 @@ -221,11 +221,6 @@ :mod:`test.regrtest`. -.. data:: have_unicode - - :const:`True` when Unicode support is available. - - .. data:: is_jython :const:`True` if the running interpreter is Jython. Modified: python/branches/py3k/Lib/test/README ============================================================================== --- python/branches/py3k/Lib/test/README (original) +++ python/branches/py3k/Lib/test/README Mon May 26 19:44:33 2008 @@ -355,8 +355,6 @@ mode, and it raises ``TestFailed`` on failure instead of ``AssertionError``. - * ``have_unicode`` - true if Unicode is available, false otherwise. - * ``is_jython`` - true if the interpreter is Jython, false otherwise. * ``TESTFN`` - a string that should always be used as the filename when Modified: python/branches/py3k/Lib/test/support.py ============================================================================== --- python/branches/py3k/Lib/test/support.py (original) +++ python/branches/py3k/Lib/test/support.py Mon May 26 19:44:33 2008 @@ -17,14 +17,14 @@ "verbose", "use_resources", "max_memuse", "record_original_stdout", "get_original_stdout", "unload", "unlink", "rmtree", "forget", "is_resource_enabled", "requires", "find_unused_port", "bind_port", - "fcmp", "have_unicode", "is_jython", "TESTFN", "HOST", "FUZZ", - "findfile", "verify", "vereq", "sortdict", "check_syntax_error", - "open_urlresource", "WarningMessage", "catch_warning", "CleanImport", - "EnvironmentVarGuard", "TransientResource", "captured_output", - "captured_stdout", "TransientResource", "transient_internet", - "run_with_locale", "set_memlimit", "bigmemtest", "bigaddrspacetest", - "BasicTestRunner", "run_unittest", "run_doctest", "threading_setup", - "threading_cleanup", "reap_children"] + "fcmp", "is_jython", "TESTFN", "HOST", "FUZZ", "findfile", "verify", + "vereq", "sortdict", "check_syntax_error", "open_urlresource", + "WarningMessage", "catch_warning", "CleanImport", "EnvironmentVarGuard", + "TransientResource", "captured_output", "captured_stdout", + "TransientResource", "transient_internet", "run_with_locale", + "set_memlimit", "bigmemtest", "bigaddrspacetest", "BasicTestRunner", + "run_unittest", "run_doctest", "threading_setup", "threading_cleanup", + "reap_children"] class Error(Exception): """Base class for regression test exceptions.""" @@ -243,12 +243,6 @@ return (len(x) > len(y)) - (len(x) < len(y)) return (x > y) - (x < y) -try: - str - have_unicode = True -except NameError: - have_unicode = False - is_jython = sys.platform.startswith('java') # Filename used for testing Modified: python/branches/py3k/Lib/test/test_float.py ============================================================================== --- python/branches/py3k/Lib/test/test_float.py (original) +++ python/branches/py3k/Lib/test/test_float.py Mon May 26 19:44:33 2008 @@ -22,11 +22,10 @@ self.assertRaises(ValueError, float, "+-3.14") self.assertRaises(ValueError, float, "-+3.14") self.assertRaises(ValueError, float, "--3.14") - if have_unicode: - self.assertEqual(float(unicode(" 3.14 ")), 3.14) - self.assertEqual(float(unicode(" \u0663.\u0661\u0664 ",'raw-unicode-escape')), 3.14) - # Implementation limitation in PyFloat_FromString() - self.assertRaises(ValueError, float, unicode("1"*10000)) + self.assertEqual(float(unicode(" 3.14 ")), 3.14) + self.assertEqual(float(unicode(" \u0663.\u0661\u0664 ",'raw-unicode-escape')), 3.14) + # Implementation limitation in PyFloat_FromString() + self.assertRaises(ValueError, float, unicode("1"*10000)) @support.run_with_locale('LC_NUMERIC', 'fr_FR', 'de_DE') def test_float_with_comma(self): From python-3000-checkins at python.org Mon May 26 19:47:11 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Mon, 26 May 2008 19:47:11 +0200 (CEST) Subject: [Python-3000-checkins] r63706 - in python/branches/py3k/Doc: library/shelve.rst reference/datamodel.rst Message-ID: <20080526174711.7016D1E4008@bag.python.org> Author: georg.brandl Date: Mon May 26 19:47:11 2008 New Revision: 63706 Log: Remove remaining references to UserDict module. Modified: python/branches/py3k/Doc/library/shelve.rst python/branches/py3k/Doc/reference/datamodel.rst Modified: python/branches/py3k/Doc/library/shelve.rst ============================================================================== --- python/branches/py3k/Doc/library/shelve.rst (original) +++ python/branches/py3k/Doc/library/shelve.rst Mon May 26 19:47:11 2008 @@ -80,8 +80,8 @@ .. class:: Shelf(dict[, protocol=None[, writeback=False]]) - A subclass of :class:`UserDict.DictMixin` which stores pickled values in the - *dict* object. + A subclass of :class:`collections.MutableMapping` which stores pickled values + in the *dict* object. By default, version 0 pickles are used to serialize values. The version of the pickle protocol can be specified with the *protocol* parameter. See the Modified: python/branches/py3k/Doc/reference/datamodel.rst ============================================================================== --- python/branches/py3k/Doc/reference/datamodel.rst (original) +++ python/branches/py3k/Doc/reference/datamodel.rst Mon May 26 19:47:11 2008 @@ -1554,26 +1554,25 @@ a sequence, the allowable keys should be the integers *k* for which ``0 <= k < N`` where *N* is the length of the sequence, or slice objects, which define a range of items. It is also recommended that mappings provide the methods -:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`, -:meth:`clear`, :meth:`setdefault`, -:meth:`pop`, :meth:`popitem`, :meth:`copy`, and +:meth:`keys`, :meth:`values`, :meth:`items`, :meth:`get`, :meth:`clear`, +:meth:`setdefault`, :meth:`pop`, :meth:`popitem`, :meth:`copy`, and :meth:`update` behaving similar to those for Python's standard dictionary -objects. The :mod:`UserDict` module provides a :class:`DictMixin` class to help -create those methods from a base set of :meth:`__getitem__`, -:meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`. Mutable sequences -should provide methods :meth:`append`, :meth:`count`, :meth:`index`, -:meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`, :meth:`reverse` and -:meth:`sort`, like Python standard list objects. Finally, sequence types should -implement addition (meaning concatenation) and multiplication (meaning -repetition) by defining the methods :meth:`__add__`, :meth:`__radd__`, -:meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and :meth:`__imul__` -described below; they should not define other numerical operators. It is -recommended that both mappings and sequences implement the :meth:`__contains__` -method to allow efficient use of the ``in`` operator; for mappings, ``in`` -should search the mapping's keys; for sequences, it should search -through the values. It is further recommended that both mappings and sequences -implement the :meth:`__iter__` method to allow efficient iteration through the -container; for mappings, :meth:`__iter__` should be the same as +objects. The :mod:`collections` module provides a :class:`MutableMapping` +abstract base class to help create those methods from a base set of +:meth:`__getitem__`, :meth:`__setitem__`, :meth:`__delitem__`, and :meth:`keys`. +Mutable sequences should provide methods :meth:`append`, :meth:`count`, +:meth:`index`, :meth:`extend`, :meth:`insert`, :meth:`pop`, :meth:`remove`, +:meth:`reverse` and :meth:`sort`, like Python standard list objects. Finally, +sequence types should implement addition (meaning concatenation) and +multiplication (meaning repetition) by defining the methods :meth:`__add__`, +:meth:`__radd__`, :meth:`__iadd__`, :meth:`__mul__`, :meth:`__rmul__` and +:meth:`__imul__` described below; they should not define other numerical +operators. It is recommended that both mappings and sequences implement the +:meth:`__contains__` method to allow efficient use of the ``in`` operator; for +mappings, ``in`` should search the mapping's keys; for sequences, it should +search through the values. It is further recommended that both mappings and +sequences implement the :meth:`__iter__` method to allow efficient iteration +through the container; for mappings, :meth:`__iter__` should be the same as :meth:`keys`; for sequences, it should iterate through the values. .. method:: object.__len__(self) From python-3000-checkins at python.org Mon May 26 19:47:51 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 19:47:51 +0200 (CEST) Subject: [Python-3000-checkins] r63707 - python/branches/py3k Message-ID: <20080526174751.950661E4008@bag.python.org> Author: benjamin.peterson Date: Mon May 26 19:47:51 2008 New Revision: 63707 Log: Blocked revisions 63704 via svnmerge ........ r63704 | benjamin.peterson | 2008-05-26 12:43:53 -0500 (Mon, 26 May 2008) | 2 lines turn PyErr_WarnPy3k into a macro ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Mon May 26 19:55:53 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Mon, 26 May 2008 19:55:53 +0200 (CEST) Subject: [Python-3000-checkins] r63708 - in python/branches/py3k/Doc: distutils/apiref.rst library/cgi.rst library/dbm.rst library/os.rst library/uu.rst Message-ID: <20080526175553.410E91E400F@bag.python.org> Author: georg.brandl Date: Mon May 26 19:55:52 2008 New Revision: 63708 Log: Fix old-style octal literals in the docs. Modified: python/branches/py3k/Doc/distutils/apiref.rst python/branches/py3k/Doc/library/cgi.rst python/branches/py3k/Doc/library/dbm.rst python/branches/py3k/Doc/library/os.rst python/branches/py3k/Doc/library/uu.rst Modified: python/branches/py3k/Doc/distutils/apiref.rst ============================================================================== --- python/branches/py3k/Doc/distutils/apiref.rst (original) +++ python/branches/py3k/Doc/distutils/apiref.rst Mon May 26 19:55:52 2008 @@ -961,7 +961,7 @@ directories. -.. function:: mkpath(name[, mode=0777, verbose=0, dry_run=0]) +.. function:: mkpath(name[, mode=0o777, verbose=0, dry_run=0]) Create a directory and any missing ancestor directories. If the directory already exists (or if *name* is the empty string, which means the current @@ -972,7 +972,7 @@ directories actually created. -.. function:: create_tree(base_dir, files[, mode=0777, verbose=0, dry_run=0]) +.. function:: create_tree(base_dir, files[, mode=0o777, verbose=0, dry_run=0]) Create all the empty directories under *base_dir* needed to put *files* there. *base_dir* is just the a name of a directory which doesn't necessarily exist Modified: python/branches/py3k/Doc/library/cgi.rst ============================================================================== --- python/branches/py3k/Doc/library/cgi.rst (original) +++ python/branches/py3k/Doc/library/cgi.rst Mon May 26 19:55:52 2008 @@ -385,7 +385,7 @@ usually this is in a directory :file:`cgi-bin` in the server tree. Make sure that your script is readable and executable by "others"; the Unix file -mode should be ``0755`` octal (use ``chmod 0755 filename``). Make sure that the +mode should be ``0o755`` octal (use ``chmod 0755 filename``). Make sure that the first line of the script contains ``#!`` starting in column 1 followed by the pathname of the Python interpreter, for instance:: @@ -394,8 +394,8 @@ Make sure the Python interpreter exists and is executable by "others". Make sure that any files your script needs to read or write are readable or -writable, respectively, by "others" --- their mode should be ``0644`` for -readable and ``0666`` for writable. This is because, for security reasons, the +writable, respectively, by "others" --- their mode should be ``0o644`` for +readable and ``0o666`` for writable. This is because, for security reasons, the HTTP server executes your script as user "nobody", without any special privileges. It can only read (write, execute) files that everybody can read (write, execute). The current directory at execution time is also different (it Modified: python/branches/py3k/Doc/library/dbm.rst ============================================================================== --- python/branches/py3k/Doc/library/dbm.rst (original) +++ python/branches/py3k/Doc/library/dbm.rst Mon May 26 19:55:52 2008 @@ -242,7 +242,7 @@ :exc:`error` is raised if an invalid flag is specified. The optional *mode* argument is the Unix mode of the file, used only when the - database has to be created. It defaults to octal ``0666``. + database has to be created. It defaults to octal ``0o666``. In addition to the dictionary-like methods, ``gdbm`` objects have the following methods: @@ -334,7 +334,7 @@ +---------+-------------------------------------------+ The optional *mode* argument is the Unix mode of the file, used only when the - database has to be created. It defaults to octal ``0666`` (and will be + database has to be created. It defaults to octal ``0o666`` (and will be modified by the prevailing umask). Modified: python/branches/py3k/Doc/library/os.rst ============================================================================== --- python/branches/py3k/Doc/library/os.rst (original) +++ python/branches/py3k/Doc/library/os.rst Mon May 26 19:55:52 2008 @@ -490,10 +490,10 @@ .. function:: open(file, flags[, mode]) - Open the file *file* and set various flags according to *flags* and possibly its - mode according to *mode*. The default *mode* is ``0777`` (octal), and the - current umask value is first masked out. Return the file descriptor for the - newly opened file. Availability: Macintosh, Unix, Windows. + Open the file *file* and set various flags according to *flags* and possibly + its mode according to *mode*. The default *mode* is ``0o777`` (octal), and + the current umask value is first masked out. Return the file descriptor for + the newly opened file. Availability: Macintosh, Unix, Windows. For a description of the flag and mode values, see the C run-time documentation; flag constants (like :const:`O_RDONLY` and :const:`O_WRONLY`) are defined in @@ -823,9 +823,9 @@ .. function:: mkfifo(path[, mode]) - Create a FIFO (a named pipe) named *path* with numeric mode *mode*. The default - *mode* is ``0666`` (octal). The current umask value is first masked out from - the mode. Availability: Macintosh, Unix. + Create a FIFO (a named pipe) named *path* with numeric mode *mode*. The + default *mode* is ``0o666`` (octal). The current umask value is first masked + out from the mode. Availability: Macintosh, Unix. FIFOs are pipes that can be accessed like regular files. FIFOs exist until they are deleted (for example with :func:`os.unlink`). Generally, FIFOs are used as @@ -834,7 +834,7 @@ doesn't open the FIFO --- it just creates the rendezvous point. -.. function:: mknod(filename[, mode=0600, device]) +.. function:: mknod(filename[, mode=0o600, device]) Create a filesystem node (file, device special file or named pipe) named *filename*. *mode* specifies both the permissions to use and the type of node to @@ -865,9 +865,10 @@ .. function:: mkdir(path[, mode]) - Create a directory named *path* with numeric mode *mode*. The default *mode* is - ``0777`` (octal). On some systems, *mode* is ignored. Where it is used, the - current umask value is first masked out. Availability: Macintosh, Unix, Windows. + Create a directory named *path* with numeric mode *mode*. The default *mode* + is ``0o777`` (octal). On some systems, *mode* is ignored. Where it is used, + the current umask value is first masked out. Availability: Macintosh, Unix, + Windows. It is also possible to create temporary directories; see the :mod:`tempfile` module's :func:`tempfile.mkdtemp` function. @@ -880,10 +881,10 @@ single: UNC paths; and os.makedirs() Recursive directory creation function. Like :func:`mkdir`, but makes all - intermediate-level directories needed to contain the leaf directory. Throws an - :exc:`error` exception if the leaf directory already exists or cannot be - created. The default *mode* is ``0777`` (octal). On some systems, *mode* is - ignored. Where it is used, the current umask value is first masked out. + intermediate-level directories needed to contain the leaf directory. Throws + an :exc:`error` exception if the leaf directory already exists or cannot be + created. The default *mode* is ``0o777`` (octal). On some systems, *mode* + is ignored. Where it is used, the current umask value is first masked out. .. note:: Modified: python/branches/py3k/Doc/library/uu.rst ============================================================================== --- python/branches/py3k/Doc/library/uu.rst (original) +++ python/branches/py3k/Doc/library/uu.rst Mon May 26 19:55:52 2008 @@ -27,10 +27,10 @@ .. function:: encode(in_file, out_file[, name[, mode]]) - Uuencode file *in_file* into file *out_file*. The uuencoded file will have the - header specifying *name* and *mode* as the defaults for the results of decoding - the file. The default defaults are taken from *in_file*, or ``'-'`` and ``0666`` - respectively. + Uuencode file *in_file* into file *out_file*. The uuencoded file will have + the header specifying *name* and *mode* as the defaults for the results of + decoding the file. The default defaults are taken from *in_file*, or ``'-'`` + and ``0o666`` respectively. .. function:: decode(in_file[, out_file[, mode[, quiet]]]) From python-3000-checkins at python.org Mon May 26 19:56:51 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Mon, 26 May 2008 19:56:51 +0200 (CEST) Subject: [Python-3000-checkins] r63709 - in python/branches/py3k/Lib: _LWPCookieJar.py _MozillaCookieJar.py http/cookiejar.py Message-ID: <20080526175651.F29131E401D@bag.python.org> Author: georg.brandl Date: Mon May 26 19:56:51 2008 New Revision: 63709 Log: Remove _FooCookieJar modules and merge their content into http.cookiejar. Removed: python/branches/py3k/Lib/_LWPCookieJar.py python/branches/py3k/Lib/_MozillaCookieJar.py Modified: python/branches/py3k/Lib/http/cookiejar.py Deleted: python/branches/py3k/Lib/_LWPCookieJar.py ============================================================================== --- python/branches/py3k/Lib/_LWPCookieJar.py Mon May 26 19:56:51 2008 +++ (empty file) @@ -1,169 +0,0 @@ -"""Load / save to libwww-perl (LWP) format files. - -Actually, the format is slightly extended from that used by LWP's -(libwww-perl's) HTTP::Cookies, to avoid losing some RFC 2965 information -not recorded by LWP. - -It uses the version string "2.0", though really there isn't an LWP Cookies -2.0 format. This indicates that there is extra information in here -(domain_dot and # port_spec) while still being compatible with -libwww-perl, I hope. - -""" - -import time, re -from http.cookiejar import (_warn_unhandled_exception, FileCookieJar, LoadError, - Cookie, MISSING_FILENAME_TEXT, - join_header_words, split_header_words, - iso2time, time2isoz) - -def lwp_cookie_str(cookie): - """Return string representation of Cookie in an the LWP cookie file format. - - Actually, the format is extended a bit -- see module docstring. - - """ - h = [(cookie.name, cookie.value), - ("path", cookie.path), - ("domain", cookie.domain)] - if cookie.port is not None: h.append(("port", cookie.port)) - if cookie.path_specified: h.append(("path_spec", None)) - if cookie.port_specified: h.append(("port_spec", None)) - if cookie.domain_initial_dot: h.append(("domain_dot", None)) - if cookie.secure: h.append(("secure", None)) - if cookie.expires: h.append(("expires", - time2isoz(float(cookie.expires)))) - if cookie.discard: h.append(("discard", None)) - if cookie.comment: h.append(("comment", cookie.comment)) - if cookie.comment_url: h.append(("commenturl", cookie.comment_url)) - - keys = sorted(cookie._rest.keys()) - for k in keys: - h.append((k, str(cookie._rest[k]))) - - h.append(("version", str(cookie.version))) - - return join_header_words([h]) - -class LWPCookieJar(FileCookieJar): - """ - The LWPCookieJar saves a sequence of"Set-Cookie3" lines. - "Set-Cookie3" is the format used by the libwww-perl libary, not known - to be compatible with any browser, but which is easy to read and - doesn't lose information about RFC 2965 cookies. - - Additional methods - - as_lwp_str(ignore_discard=True, ignore_expired=True) - - """ - - def as_lwp_str(self, ignore_discard=True, ignore_expires=True): - """Return cookies as a string of "\n"-separated "Set-Cookie3" headers. - - ignore_discard and ignore_expires: see docstring for FileCookieJar.save - - """ - now = time.time() - r = [] - for cookie in self: - if not ignore_discard and cookie.discard: - continue - if not ignore_expires and cookie.is_expired(now): - continue - r.append("Set-Cookie3: %s" % lwp_cookie_str(cookie)) - return "\n".join(r+[""]) - - def save(self, filename=None, ignore_discard=False, ignore_expires=False): - if filename is None: - if self.filename is not None: filename = self.filename - else: raise ValueError(MISSING_FILENAME_TEXT) - - f = open(filename, "w") - try: - # There really isn't an LWP Cookies 2.0 format, but this indicates - # that there is extra information in here (domain_dot and - # port_spec) while still being compatible with libwww-perl, I hope. - f.write("#LWP-Cookies-2.0\n") - f.write(self.as_lwp_str(ignore_discard, ignore_expires)) - finally: - f.close() - - def _really_load(self, f, filename, ignore_discard, ignore_expires): - magic = f.readline() - if not re.search(self.magic_re, magic): - msg = ("%r does not look like a Set-Cookie3 (LWP) format " - "file" % filename) - raise LoadError(msg) - - now = time.time() - - header = "Set-Cookie3:" - boolean_attrs = ("port_spec", "path_spec", "domain_dot", - "secure", "discard") - value_attrs = ("version", - "port", "path", "domain", - "expires", - "comment", "commenturl") - - try: - while 1: - line = f.readline() - if line == "": break - if not line.startswith(header): - continue - line = line[len(header):].strip() - - for data in split_header_words([line]): - name, value = data[0] - standard = {} - rest = {} - for k in boolean_attrs: - standard[k] = False - for k, v in data[1:]: - if k is not None: - lc = k.lower() - else: - lc = None - # don't lose case distinction for unknown fields - if (lc in value_attrs) or (lc in boolean_attrs): - k = lc - if k in boolean_attrs: - if v is None: v = True - standard[k] = v - elif k in value_attrs: - standard[k] = v - else: - rest[k] = v - - h = standard.get - expires = h("expires") - discard = h("discard") - if expires is not None: - expires = iso2time(expires) - if expires is None: - discard = True - domain = h("domain") - domain_specified = domain.startswith(".") - c = Cookie(h("version"), name, value, - h("port"), h("port_spec"), - domain, domain_specified, h("domain_dot"), - h("path"), h("path_spec"), - h("secure"), - expires, - discard, - h("comment"), - h("commenturl"), - rest) - if not ignore_discard and c.discard: - continue - if not ignore_expires and c.is_expired(now): - continue - self.set_cookie(c) - - except IOError: - raise - except Exception: - _warn_unhandled_exception() - raise LoadError("invalid Set-Cookie3 format file %r: %r" % - (filename, line)) Deleted: python/branches/py3k/Lib/_MozillaCookieJar.py ============================================================================== --- python/branches/py3k/Lib/_MozillaCookieJar.py Mon May 26 19:56:51 2008 +++ (empty file) @@ -1,149 +0,0 @@ -"""Mozilla / Netscape cookie loading / saving.""" - -import re, time - -from http.cookiejar import (_warn_unhandled_exception, FileCookieJar, LoadError, - Cookie, MISSING_FILENAME_TEXT) - -class MozillaCookieJar(FileCookieJar): - """ - - WARNING: you may want to backup your browser's cookies file if you use - this class to save cookies. I *think* it works, but there have been - bugs in the past! - - This class differs from CookieJar only in the format it uses to save and - load cookies to and from a file. This class uses the Mozilla/Netscape - `cookies.txt' format. lynx uses this file format, too. - - Don't expect cookies saved while the browser is running to be noticed by - the browser (in fact, Mozilla on unix will overwrite your saved cookies if - you change them on disk while it's running; on Windows, you probably can't - save at all while the browser is running). - - Note that the Mozilla/Netscape format will downgrade RFC2965 cookies to - Netscape cookies on saving. - - In particular, the cookie version and port number information is lost, - together with information about whether or not Path, Port and Discard were - specified by the Set-Cookie2 (or Set-Cookie) header, and whether or not the - domain as set in the HTTP header started with a dot (yes, I'm aware some - domains in Netscape files start with a dot and some don't -- trust me, you - really don't want to know any more about this). - - Note that though Mozilla and Netscape use the same format, they use - slightly different headers. The class saves cookies using the Netscape - header by default (Mozilla can cope with that). - - """ - magic_re = "#( Netscape)? HTTP Cookie File" - header = """\ - # Netscape HTTP Cookie File - # http://www.netscape.com/newsref/std/cookie_spec.html - # This is a generated file! Do not edit. - -""" - - def _really_load(self, f, filename, ignore_discard, ignore_expires): - now = time.time() - - magic = f.readline() - if not re.search(self.magic_re, magic): - f.close() - raise LoadError( - "%r does not look like a Netscape format cookies file" % - filename) - - try: - while 1: - line = f.readline() - if line == "": break - - # last field may be absent, so keep any trailing tab - if line.endswith("\n"): line = line[:-1] - - # skip comments and blank lines XXX what is $ for? - if (line.strip().startswith(("#", "$")) or - line.strip() == ""): - continue - - domain, domain_specified, path, secure, expires, name, value = \ - line.split("\t") - secure = (secure == "TRUE") - domain_specified = (domain_specified == "TRUE") - if name == "": - # cookies.txt regards 'Set-Cookie: foo' as a cookie - # with no name, whereas http.cookiejar regards it as a - # cookie with no value. - name = value - value = None - - initial_dot = domain.startswith(".") - assert domain_specified == initial_dot - - discard = False - if expires == "": - expires = None - discard = True - - # assume path_specified is false - c = Cookie(0, name, value, - None, False, - domain, domain_specified, initial_dot, - path, False, - secure, - expires, - discard, - None, - None, - {}) - if not ignore_discard and c.discard: - continue - if not ignore_expires and c.is_expired(now): - continue - self.set_cookie(c) - - except IOError: - raise - except Exception: - _warn_unhandled_exception() - raise LoadError("invalid Netscape format cookies file %r: %r" % - (filename, line)) - - def save(self, filename=None, ignore_discard=False, ignore_expires=False): - if filename is None: - if self.filename is not None: filename = self.filename - else: raise ValueError(MISSING_FILENAME_TEXT) - - f = open(filename, "w") - try: - f.write(self.header) - now = time.time() - for cookie in self: - if not ignore_discard and cookie.discard: - continue - if not ignore_expires and cookie.is_expired(now): - continue - if cookie.secure: secure = "TRUE" - else: secure = "FALSE" - if cookie.domain.startswith("."): initial_dot = "TRUE" - else: initial_dot = "FALSE" - if cookie.expires is not None: - expires = str(cookie.expires) - else: - expires = "" - if cookie.value is None: - # cookies.txt regards 'Set-Cookie: foo' as a cookie - # with no name, whereas http.cookiejar regards it as a - # cookie with no value. - name = "" - value = cookie.name - else: - name = cookie.name - value = cookie.value - f.write( - "\t".join([cookie.domain, initial_dot, cookie.path, - secure, expires, name, value])+ - "\n") - finally: - f.close() Modified: python/branches/py3k/Lib/http/cookiejar.py ============================================================================== --- python/branches/py3k/Lib/http/cookiejar.py (original) +++ python/branches/py3k/Lib/http/cookiejar.py Mon May 26 19:56:51 2008 @@ -1781,5 +1781,298 @@ finally: self._cookies_lock.release() -from _LWPCookieJar import LWPCookieJar, lwp_cookie_str -from _MozillaCookieJar import MozillaCookieJar + +def lwp_cookie_str(cookie): + """Return string representation of Cookie in an the LWP cookie file format. + + Actually, the format is extended a bit -- see module docstring. + + """ + h = [(cookie.name, cookie.value), + ("path", cookie.path), + ("domain", cookie.domain)] + if cookie.port is not None: h.append(("port", cookie.port)) + if cookie.path_specified: h.append(("path_spec", None)) + if cookie.port_specified: h.append(("port_spec", None)) + if cookie.domain_initial_dot: h.append(("domain_dot", None)) + if cookie.secure: h.append(("secure", None)) + if cookie.expires: h.append(("expires", + time2isoz(float(cookie.expires)))) + if cookie.discard: h.append(("discard", None)) + if cookie.comment: h.append(("comment", cookie.comment)) + if cookie.comment_url: h.append(("commenturl", cookie.comment_url)) + + keys = sorted(cookie._rest.keys()) + for k in keys: + h.append((k, str(cookie._rest[k]))) + + h.append(("version", str(cookie.version))) + + return join_header_words([h]) + +class LWPCookieJar(FileCookieJar): + """ + The LWPCookieJar saves a sequence of"Set-Cookie3" lines. + "Set-Cookie3" is the format used by the libwww-perl libary, not known + to be compatible with any browser, but which is easy to read and + doesn't lose information about RFC 2965 cookies. + + Additional methods + + as_lwp_str(ignore_discard=True, ignore_expired=True) + + """ + + def as_lwp_str(self, ignore_discard=True, ignore_expires=True): + """Return cookies as a string of "\n"-separated "Set-Cookie3" headers. + + ignore_discard and ignore_expires: see docstring for FileCookieJar.save + + """ + now = time.time() + r = [] + for cookie in self: + if not ignore_discard and cookie.discard: + continue + if not ignore_expires and cookie.is_expired(now): + continue + r.append("Set-Cookie3: %s" % lwp_cookie_str(cookie)) + return "\n".join(r+[""]) + + def save(self, filename=None, ignore_discard=False, ignore_expires=False): + if filename is None: + if self.filename is not None: filename = self.filename + else: raise ValueError(MISSING_FILENAME_TEXT) + + f = open(filename, "w") + try: + # There really isn't an LWP Cookies 2.0 format, but this indicates + # that there is extra information in here (domain_dot and + # port_spec) while still being compatible with libwww-perl, I hope. + f.write("#LWP-Cookies-2.0\n") + f.write(self.as_lwp_str(ignore_discard, ignore_expires)) + finally: + f.close() + + def _really_load(self, f, filename, ignore_discard, ignore_expires): + magic = f.readline() + if not re.search(self.magic_re, magic): + msg = ("%r does not look like a Set-Cookie3 (LWP) format " + "file" % filename) + raise LoadError(msg) + + now = time.time() + + header = "Set-Cookie3:" + boolean_attrs = ("port_spec", "path_spec", "domain_dot", + "secure", "discard") + value_attrs = ("version", + "port", "path", "domain", + "expires", + "comment", "commenturl") + + try: + while 1: + line = f.readline() + if line == "": break + if not line.startswith(header): + continue + line = line[len(header):].strip() + + for data in split_header_words([line]): + name, value = data[0] + standard = {} + rest = {} + for k in boolean_attrs: + standard[k] = False + for k, v in data[1:]: + if k is not None: + lc = k.lower() + else: + lc = None + # don't lose case distinction for unknown fields + if (lc in value_attrs) or (lc in boolean_attrs): + k = lc + if k in boolean_attrs: + if v is None: v = True + standard[k] = v + elif k in value_attrs: + standard[k] = v + else: + rest[k] = v + + h = standard.get + expires = h("expires") + discard = h("discard") + if expires is not None: + expires = iso2time(expires) + if expires is None: + discard = True + domain = h("domain") + domain_specified = domain.startswith(".") + c = Cookie(h("version"), name, value, + h("port"), h("port_spec"), + domain, domain_specified, h("domain_dot"), + h("path"), h("path_spec"), + h("secure"), + expires, + discard, + h("comment"), + h("commenturl"), + rest) + if not ignore_discard and c.discard: + continue + if not ignore_expires and c.is_expired(now): + continue + self.set_cookie(c) + + except IOError: + raise + except Exception: + _warn_unhandled_exception() + raise LoadError("invalid Set-Cookie3 format file %r: %r" % + (filename, line)) + + +class MozillaCookieJar(FileCookieJar): + """ + + WARNING: you may want to backup your browser's cookies file if you use + this class to save cookies. I *think* it works, but there have been + bugs in the past! + + This class differs from CookieJar only in the format it uses to save and + load cookies to and from a file. This class uses the Mozilla/Netscape + `cookies.txt' format. lynx uses this file format, too. + + Don't expect cookies saved while the browser is running to be noticed by + the browser (in fact, Mozilla on unix will overwrite your saved cookies if + you change them on disk while it's running; on Windows, you probably can't + save at all while the browser is running). + + Note that the Mozilla/Netscape format will downgrade RFC2965 cookies to + Netscape cookies on saving. + + In particular, the cookie version and port number information is lost, + together with information about whether or not Path, Port and Discard were + specified by the Set-Cookie2 (or Set-Cookie) header, and whether or not the + domain as set in the HTTP header started with a dot (yes, I'm aware some + domains in Netscape files start with a dot and some don't -- trust me, you + really don't want to know any more about this). + + Note that though Mozilla and Netscape use the same format, they use + slightly different headers. The class saves cookies using the Netscape + header by default (Mozilla can cope with that). + + """ + magic_re = "#( Netscape)? HTTP Cookie File" + header = """\ + # Netscape HTTP Cookie File + # http://www.netscape.com/newsref/std/cookie_spec.html + # This is a generated file! Do not edit. + +""" + + def _really_load(self, f, filename, ignore_discard, ignore_expires): + now = time.time() + + magic = f.readline() + if not re.search(self.magic_re, magic): + f.close() + raise LoadError( + "%r does not look like a Netscape format cookies file" % + filename) + + try: + while 1: + line = f.readline() + if line == "": break + + # last field may be absent, so keep any trailing tab + if line.endswith("\n"): line = line[:-1] + + # skip comments and blank lines XXX what is $ for? + if (line.strip().startswith(("#", "$")) or + line.strip() == ""): + continue + + domain, domain_specified, path, secure, expires, name, value = \ + line.split("\t") + secure = (secure == "TRUE") + domain_specified = (domain_specified == "TRUE") + if name == "": + # cookies.txt regards 'Set-Cookie: foo' as a cookie + # with no name, whereas http.cookiejar regards it as a + # cookie with no value. + name = value + value = None + + initial_dot = domain.startswith(".") + assert domain_specified == initial_dot + + discard = False + if expires == "": + expires = None + discard = True + + # assume path_specified is false + c = Cookie(0, name, value, + None, False, + domain, domain_specified, initial_dot, + path, False, + secure, + expires, + discard, + None, + None, + {}) + if not ignore_discard and c.discard: + continue + if not ignore_expires and c.is_expired(now): + continue + self.set_cookie(c) + + except IOError: + raise + except Exception: + _warn_unhandled_exception() + raise LoadError("invalid Netscape format cookies file %r: %r" % + (filename, line)) + + def save(self, filename=None, ignore_discard=False, ignore_expires=False): + if filename is None: + if self.filename is not None: filename = self.filename + else: raise ValueError(MISSING_FILENAME_TEXT) + + f = open(filename, "w") + try: + f.write(self.header) + now = time.time() + for cookie in self: + if not ignore_discard and cookie.discard: + continue + if not ignore_expires and cookie.is_expired(now): + continue + if cookie.secure: secure = "TRUE" + else: secure = "FALSE" + if cookie.domain.startswith("."): initial_dot = "TRUE" + else: initial_dot = "FALSE" + if cookie.expires is not None: + expires = str(cookie.expires) + else: + expires = "" + if cookie.value is None: + # cookies.txt regards 'Set-Cookie: foo' as a cookie + # with no name, whereas http.cookiejar regards it as a + # cookie with no value. + name = "" + value = cookie.name + else: + name = cookie.name + value = cookie.value + f.write( + "\t".join([cookie.domain, initial_dot, cookie.path, + secure, expires, name, value])+ + "\n") + finally: + f.close() From python-3000-checkins at python.org Mon May 26 20:03:26 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Mon, 26 May 2008 20:03:26 +0200 (CEST) Subject: [Python-3000-checkins] r63710 - python/branches/py3k/Misc/NEWS Message-ID: <20080526180326.84D141E4250@bag.python.org> Author: georg.brandl Date: Mon May 26 20:03:26 2008 New Revision: 63710 Log: Add NEWS entry for http package. Modified: python/branches/py3k/Misc/NEWS Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Mon May 26 20:03:26 2008 @@ -56,6 +56,11 @@ Library ------- +- The ``http`` package was created; it contains the old ``httplib`` + as ``http.client``, ``Cookie`` as ``http.cookies``, ``cookielib`` + as ``http.cookiejar``, and the content of the three ``HTTPServer`` + modules as ``http.server``. + - The ``xmlrpc`` package was created; it contains the old ``xmlrpclib`` module as ``xmlrpc.client`` and the content of the old ``SimpleXMLRPCServer`` and ``DocXMLRPCServer`` modules From python-3000-checkins at python.org Mon May 26 20:54:30 2008 From: python-3000-checkins at python.org (brett.cannon) Date: Mon, 26 May 2008 20:54:30 +0200 (CEST) Subject: [Python-3000-checkins] r63712 - in python/branches/py3k/Lib: dbm html http tkinter xmlrpc Message-ID: <20080526185430.DF0031E468A@bag.python.org> Author: brett.cannon Date: Mon May 26 20:54:30 2008 New Revision: 63712 Log: Ignore py(c|o) files. Modified: python/branches/py3k/Lib/dbm/ (props changed) python/branches/py3k/Lib/html/ (props changed) python/branches/py3k/Lib/http/ (props changed) python/branches/py3k/Lib/tkinter/ (props changed) python/branches/py3k/Lib/xmlrpc/ (props changed) From python-3000-checkins at python.org Mon May 26 20:56:21 2008 From: python-3000-checkins at python.org (brett.cannon) Date: Mon, 26 May 2008 20:56:21 +0200 (CEST) Subject: [Python-3000-checkins] r63713 - python/branches/py3k/Makefile.pre.in Message-ID: <20080526185621.BFE551E467A@bag.python.org> Author: brett.cannon Date: Mon May 26 20:56:21 2008 New Revision: 63713 Log: Add the new packages to the Makefile for installation. Modified: python/branches/py3k/Makefile.pre.in Modified: python/branches/py3k/Makefile.pre.in ============================================================================== --- python/branches/py3k/Makefile.pre.in (original) +++ python/branches/py3k/Makefile.pre.in Mon May 26 20:56:21 2008 @@ -818,7 +818,7 @@ test/decimaltestdata \ encodings \ email email/mime email/test email/test/data \ - html json json/tests \ + html json json/tests http dbm xmlrpc \ sqlite3 sqlite3/test \ logging bsddb bsddb/test csv wsgiref \ lib2to3 lib2to3/fixes lib2to3/pgen2 lib2to3/tests \ From python-3000-checkins at python.org Mon May 26 21:04:21 2008 From: python-3000-checkins at python.org (brett.cannon) Date: Mon, 26 May 2008 21:04:21 +0200 (CEST) Subject: [Python-3000-checkins] r63715 - in python/branches/py3k: Demo/tkinter/guido/kill.py Demo/tkinter/guido/svkill.py Doc/library/commands.rst Doc/library/subprocess.rst Doc/library/unix.rst Lib/commands.py Lib/subprocess.py Lib/test/regrtest.py Lib/test/test___all__.py Lib/test/test_commands.py Lib/test/test_subprocess.py Misc/NEWS Misc/cheatsheet Message-ID: <20080526190421.CFC5A1E401B@bag.python.org> Author: brett.cannon Date: Mon May 26 21:04:21 2008 New Revision: 63715 Log: The commands module has been removed. The getoutput() and getstatusoutput() functions have been added to the subprocess module. The fixer for this still needs to be written and proper Py3K deprecation warnings for the functions that didn't make the transition need to be done in 2.6. This is all part of trying to close issue #2872. Removed: python/branches/py3k/Doc/library/commands.rst python/branches/py3k/Lib/commands.py python/branches/py3k/Lib/test/test_commands.py Modified: python/branches/py3k/Demo/tkinter/guido/kill.py python/branches/py3k/Demo/tkinter/guido/svkill.py python/branches/py3k/Doc/library/subprocess.rst python/branches/py3k/Doc/library/unix.rst python/branches/py3k/Lib/subprocess.py python/branches/py3k/Lib/test/regrtest.py python/branches/py3k/Lib/test/test___all__.py python/branches/py3k/Lib/test/test_subprocess.py python/branches/py3k/Misc/NEWS python/branches/py3k/Misc/cheatsheet Modified: python/branches/py3k/Demo/tkinter/guido/kill.py ============================================================================== --- python/branches/py3k/Demo/tkinter/guido/kill.py (original) +++ python/branches/py3k/Demo/tkinter/guido/kill.py Mon May 26 21:04:21 2008 @@ -4,7 +4,7 @@ from Tkinter import * from string import splitfields from string import split -import commands +import subprocess import os class BarButton(Menubutton): @@ -31,7 +31,7 @@ self.do_update() def do_update(self): name, option, column = self.format_list[self.format.get()] - s = commands.getoutput('ps -w ' + option) + s = subprocess.getoutput('ps -w ' + option) list = splitfields(s, '\n') self.header.set(list[0]) del list[0] Modified: python/branches/py3k/Demo/tkinter/guido/svkill.py ============================================================================== --- python/branches/py3k/Demo/tkinter/guido/svkill.py (original) +++ python/branches/py3k/Demo/tkinter/guido/svkill.py Mon May 26 21:04:21 2008 @@ -9,7 +9,7 @@ from string import splitfields from string import split -import commands +import subprocess import os user = os.environ['LOGNAME'] @@ -46,7 +46,7 @@ def do_update(self): format = self.format_list[self.format.get()][1] view = self.view_list[self.view.get()][1] - s = commands.getoutput('ps %s %s' % (view, format)) + s = subprocess.getoutput('ps %s %s' % (view, format)) list = splitfields(s, '\n') self.header.set(list[0] + ' ') del list[0] Deleted: python/branches/py3k/Doc/library/commands.rst ============================================================================== --- python/branches/py3k/Doc/library/commands.rst Mon May 26 21:04:21 2008 +++ (empty file) @@ -1,53 +0,0 @@ - -:mod:`commands` --- Utilities for running commands -================================================== - -.. module:: commands - :platform: Unix - :synopsis: Utility functions for running external commands. -.. sectionauthor:: Sue Williams - - -The :mod:`commands` module contains wrapper functions for :func:`os.popen` which -take a system command as a string and return any output generated by the command -and, optionally, the exit status. - -The :mod:`subprocess` module provides more powerful facilities for spawning new -processes and retrieving their results. Using the :mod:`subprocess` module is -preferable to using the :mod:`commands` module. - -The :mod:`commands` module defines the following functions: - - -.. function:: getstatusoutput(cmd) - - Execute the string *cmd* in a shell with :func:`os.popen` and return a 2-tuple - ``(status, output)``. *cmd* is actually run as ``{ cmd ; } 2>&1``, so that the - returned output will contain output or error messages. A trailing newline is - stripped from the output. The exit status for the command can be interpreted - according to the rules for the C function :cfunc:`wait`. - - -.. function:: getoutput(cmd) - - Like :func:`getstatusoutput`, except the exit status is ignored and the return - value is a string containing the command's output. - -Example:: - - >>> import commands - >>> commands.getstatusoutput('ls /bin/ls') - (0, '/bin/ls') - >>> commands.getstatusoutput('cat /bin/junk') - (256, 'cat: /bin/junk: No such file or directory') - >>> commands.getstatusoutput('/bin/junk') - (256, 'sh: /bin/junk: not found') - >>> commands.getoutput('ls /bin/ls') - '/bin/ls' - - -.. seealso:: - - Module :mod:`subprocess` - Module for spawning and managing subprocesses. - Modified: python/branches/py3k/Doc/library/subprocess.rst ============================================================================== --- python/branches/py3k/Doc/library/subprocess.rst (original) +++ python/branches/py3k/Doc/library/subprocess.rst Mon May 26 21:04:21 2008 @@ -14,7 +14,6 @@ os.system os.spawn* - commands.* Information about how the :mod:`subprocess` module can be used to replace these modules and functions can be found in the following sections. @@ -113,7 +112,7 @@ Convenience Functions ^^^^^^^^^^^^^^^^^^^^^ -This module also defines two shortcut functions: +This module also defines four shortcut functions: .. function:: call(*popenargs, **kwargs) @@ -138,6 +137,35 @@ check_call(["ls", "-l"]) +.. function:: getstatusoutput(cmd) + Return ``(status, output)`` of executing *cmd* in a shell. + + Execute the string *cmd* in a shell with :func:`os.popen` and return a 2-tuple + ``(status, output)``. *cmd* is actually run as ``{ cmd ; } 2>&1``, so that the + returned output will contain output or error messages. A trailing newline is + stripped from the output. The exit status for the command can be interpreted + according to the rules for the C function :cfunc:`wait`. Example:: + + >>> import subprocess + >>> subprocess.getstatusoutput('ls /bin/ls') + (0, '/bin/ls') + >>> subprocess.getstatusoutput('cat /bin/junk') + (256, 'cat: /bin/junk: No such file or directory') + >>> subprocess.getstatusoutput('/bin/junk') + (256, 'sh: /bin/junk: not found') + + +.. function:: getoutput(cmd) + Return output ``(stdout or stderr)`` of executing *cmd* in a shell. + + Like :func:`getstatusoutput`, except the exit status is ignored and the return + value is a string containing the command's output. Example:: + + >>> import subprocess + >>> subprocess.getoutput('ls /bin/ls') + '/bin/ls' + + Exceptions ^^^^^^^^^^ Modified: python/branches/py3k/Doc/library/unix.rst ============================================================================== --- python/branches/py3k/Doc/library/unix.rst (original) +++ python/branches/py3k/Doc/library/unix.rst Mon May 26 21:04:21 2008 @@ -25,4 +25,3 @@ resource.rst nis.rst syslog.rst - commands.rst Deleted: python/branches/py3k/Lib/commands.py ============================================================================== --- python/branches/py3k/Lib/commands.py Mon May 26 21:04:21 2008 +++ (empty file) @@ -1,77 +0,0 @@ -"""Execute shell commands via os.popen() and return status, output. - -Interface summary: - - import commands - - outtext = commands.getoutput(cmd) - (exitstatus, outtext) = commands.getstatusoutput(cmd) - outtext = commands.getstatus(file) # returns output of "ls -ld file" - -A trailing newline is removed from the output string. - -Encapsulates the basic operation: - - pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r') - text = pipe.read() - sts = pipe.close() - - [Note: it would be nice to add functions to interpret the exit status.] -""" - -__all__ = ["getstatusoutput", "getoutput"] - -# Module 'commands' -# -# Various tools for executing commands and looking at their output and status. -# -# NB This only works (and is only relevant) for UNIX. - - -# Get the output from a shell command into a string. -# The exit status is ignored; a trailing newline is stripped. -# Assume the command will work with '{ ... ; } 2>&1' around it.. -# -def getoutput(cmd): - """Return output (stdout or stderr) of executing cmd in a shell.""" - return getstatusoutput(cmd)[1] - - -# Ditto but preserving the exit status. -# Returns a pair (sts, output) -# -def getstatusoutput(cmd): - """Return (status, output) of executing cmd in a shell.""" - import os - pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r') - text = pipe.read() - sts = pipe.close() - if sts is None: sts = 0 - if text[-1:] == '\n': text = text[:-1] - return sts, text - - -# Make command argument from directory and pathname (prefix space, add quotes). -# -def mk2arg(head, x): - import os - return mkarg(os.path.join(head, x)) - - -# Make a shell command argument from a string. -# Return a string beginning with a space followed by a shell-quoted -# version of the argument. -# Two strategies: enclose in single quotes if it contains none; -# otherwise, enclose in double quotes and prefix quotable characters -# with backslash. -# -def mkarg(x): - if '\'' not in x: - return ' \'' + x + '\'' - s = ' "' - for c in x: - if c in '\\$"`': - s = s + '\\' - s = s + c - s = s + '"' - return s Modified: python/branches/py3k/Lib/subprocess.py ============================================================================== --- python/branches/py3k/Lib/subprocess.py (original) +++ python/branches/py3k/Lib/subprocess.py Mon May 26 21:04:21 2008 @@ -17,7 +17,6 @@ os.system os.spawn* -commands.* Information about how the subprocess module can be used to replace these modules and functions can be found below. @@ -105,7 +104,7 @@ (Windows only) -This module also defines two shortcut functions: +This module also defines four shortcut functions: call(*popenargs, **kwargs): Run command with arguments. Wait for command to complete, then @@ -125,6 +124,34 @@ check_call(["ls", "-l"]) +getstatusoutput(cmd): + Return (status, output) of executing cmd in a shell. + + Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple + (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the + returned output will contain output or error messages. A trailing newline + is stripped from the output. The exit status for the command can be + interpreted according to the rules for the C function wait(). Example: + + >>> import subprocess + >>> subprocess.getstatusoutput('ls /bin/ls') + (0, '/bin/ls') + >>> subprocess.getstatusoutput('cat /bin/junk') + (256, 'cat: /bin/junk: No such file or directory') + >>> subprocess.getstatusoutput('/bin/junk') + (256, 'sh: /bin/junk: not found') + +getoutput(cmd): + Return output (stdout or stderr) of executing cmd in a shell. + + Like getstatusoutput(), except the exit status is ignored and the return + value is a string containing the command's output. Example: + + >>> import subprocess + >>> subprocess.getoutput('ls /bin/ls') + '/bin/ls' + + Exceptions ---------- Exceptions raised in the child process, before the new program has @@ -336,7 +363,8 @@ import fcntl import pickle -__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"] +__all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "getstatusoutput", + "getoutput", "CalledProcessError"] try: MAXFD = os.sysconf("SC_OPEN_MAX") @@ -458,6 +486,48 @@ return ''.join(result) +# Various tools for executing commands and looking at their output and status. +# +# NB This only works (and is only relevant) for UNIX. + +def getstatusoutput(cmd): + """Return (status, output) of executing cmd in a shell. + + Execute the string 'cmd' in a shell with os.popen() and return a 2-tuple + (status, output). cmd is actually run as '{ cmd ; } 2>&1', so that the + returned output will contain output or error messages. A trailing newline + is stripped from the output. The exit status for the command can be + interpreted according to the rules for the C function wait(). Example: + + >>> import subprocess + >>> subprocess.getstatusoutput('ls /bin/ls') + (0, '/bin/ls') + >>> subprocess.getstatusoutput('cat /bin/junk') + (256, 'cat: /bin/junk: No such file or directory') + >>> subprocess.getstatusoutput('/bin/junk') + (256, 'sh: /bin/junk: not found') + """ + pipe = os.popen('{ ' + cmd + '; } 2>&1', 'r') + text = pipe.read() + sts = pipe.close() + if sts is None: sts = 0 + if text[-1:] == '\n': text = text[:-1] + return sts, text + + +def getoutput(cmd): + """Return output (stdout or stderr) of executing cmd in a shell. + + Like getstatusoutput(), except the exit status is ignored and the return + value is a string containing the command's output. Example: + + >>> import subprocess + >>> subprocess.getoutput('ls /bin/ls') + '/bin/ls' + """ + return getstatusoutput(cmd)[1] + + class Popen(object): def __init__(self, args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, Modified: python/branches/py3k/Lib/test/regrtest.py ============================================================================== --- python/branches/py3k/Lib/test/regrtest.py (original) +++ python/branches/py3k/Lib/test/regrtest.py Mon May 26 21:04:21 2008 @@ -868,7 +868,6 @@ """ test__locale test_bsddb3 - test_commands test_crypt test_curses test_dbm @@ -910,7 +909,6 @@ test_bsddb test_bsddb3 test_bz2 - test_commands test_crypt test_curses test_dbm @@ -1060,7 +1058,6 @@ """ test_audioop test_bsddb3 - test_commands test_curses test_dl test_epoll Modified: python/branches/py3k/Lib/test/test___all__.py ============================================================================== --- python/branches/py3k/Lib/test/test___all__.py (original) +++ python/branches/py3k/Lib/test/test___all__.py Mon May 26 21:04:21 2008 @@ -50,7 +50,6 @@ self.check_all("codecs") self.check_all("codeop") self.check_all("colorsys") - self.check_all("commands") self.check_all("compileall") self.check_all("copy") self.check_all("copyreg") Deleted: python/branches/py3k/Lib/test/test_commands.py ============================================================================== --- python/branches/py3k/Lib/test/test_commands.py Mon May 26 21:04:21 2008 +++ (empty file) @@ -1,47 +0,0 @@ -''' - Tests for commands module - Nick Mathewson -''' -import unittest -import os, tempfile, re - -from test.support import TestSkipped, run_unittest, reap_children -from commands import * - -# The module says: -# "NB This only works (and is only relevant) for UNIX." -# -# Actually, getoutput should work on any platform with an os.popen, but -# I'll take the comment as given, and skip this suite. - -if os.name != 'posix': - raise TestSkipped('Not posix; skipping test_commands') - - -class CommandTests(unittest.TestCase): - - def test_getoutput(self): - self.assertEquals(getoutput('echo xyzzy'), 'xyzzy') - self.assertEquals(getstatusoutput('echo xyzzy'), (0, 'xyzzy')) - - # we use mkdtemp in the next line to create an empty directory - # under our exclusive control; from that, we can invent a pathname - # that we _know_ won't exist. This is guaranteed to fail. - dir = None - try: - dir = tempfile.mkdtemp() - name = os.path.join(dir, "foo") - - status, output = getstatusoutput('cat ' + name) - self.assertNotEquals(status, 0) - finally: - if dir is not None: - os.rmdir(dir) - - -def test_main(): - run_unittest(CommandTests) - reap_children() - -if __name__ == "__main__": - test_main() Modified: python/branches/py3k/Lib/test/test_subprocess.py ============================================================================== --- python/branches/py3k/Lib/test/test_subprocess.py (original) +++ python/branches/py3k/Lib/test/test_subprocess.py Mon May 26 21:04:21 2008 @@ -702,10 +702,36 @@ p.terminate() self.assertNotEqual(p.wait(), 0) +class CommandTests(unittest.TestCase): +# The module says: +# "NB This only works (and is only relevant) for UNIX." +# +# Actually, getoutput should work on any platform with an os.popen, but +# I'll take the comment as given, and skip this suite. + if os.name == 'posix': + + def test_getoutput(self): + self.assertEquals(subprocess.getoutput('echo xyzzy'), 'xyzzy') + self.assertEquals(subprocess.getstatusoutput('echo xyzzy'), + (0, 'xyzzy')) + + # we use mkdtemp in the next line to create an empty directory + # under our exclusive control; from that, we can invent a pathname + # that we _know_ won't exist. This is guaranteed to fail. + dir = None + try: + dir = tempfile.mkdtemp() + name = os.path.join(dir, "foo") + + status, output = subprocess.getstatusoutput('cat ' + name) + self.assertNotEquals(status, 0) + finally: + if dir is not None: + os.rmdir(dir) + def test_main(): - support.run_unittest(ProcessTestCase) - if hasattr(support, "reap_children"): - support.reap_children() + support.run_unittest(ProcessTestCase, CommandTests) + support.reap_children() if __name__ == "__main__": - unittest.main() # XXX test_main() + test_main() Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Mon May 26 21:04:21 2008 @@ -56,6 +56,9 @@ Library ------- +- The ``commands`` module has been removed. Its getoutput() and + getstatusoutput() functions have been moved to the ``subprocess`` module. + - The ``http`` package was created; it contains the old ``httplib`` as ``http.client``, ``Cookie`` as ``http.cookies``, ``cookielib`` as ``http.cookiejar``, and the content of the three ``HTTPServer`` Modified: python/branches/py3k/Misc/cheatsheet ============================================================================== --- python/branches/py3k/Misc/cheatsheet (original) +++ python/branches/py3k/Misc/cheatsheet Mon May 26 21:04:21 2008 @@ -1811,7 +1811,6 @@ code Utilities needed to emulate Python's interactive interpreter codecs Lookup existing Unicode encodings and register new ones. colorsys Conversion functions between RGB and other color systems. -commands Tools for executing UNIX commands . compileall Force "compilation" of all .py files in a directory. configparser Configuration file parser (much like windows .ini files) copy Generic shallow and deep copying operations. From python-3000-checkins at python.org Mon May 26 21:08:32 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 21:08:32 +0200 (CEST) Subject: [Python-3000-checkins] r63717 - python/branches/py3k/Lib/test/regrtest.py Message-ID: <20080526190832.179A41E400B@bag.python.org> Author: benjamin.peterson Date: Mon May 26 21:08:31 2008 New Revision: 63717 Log: update test ignore list Modified: python/branches/py3k/Lib/test/regrtest.py Modified: python/branches/py3k/Lib/test/regrtest.py ============================================================================== --- python/branches/py3k/Lib/test/regrtest.py (original) +++ python/branches/py3k/Lib/test/regrtest.py Mon May 26 21:08:31 2008 @@ -875,7 +875,7 @@ test_fcntl test_fork1 test_epoll - test_gdbm + test_dbm_gnu test_grp test_ioctl test_largefile @@ -989,7 +989,7 @@ test_bsddb3 test_curses test_epoll - test_gdbm + test_dbm_gnu test_largefile test_locale test_minidom @@ -1003,7 +1003,7 @@ test_dbm test_epoll test_kqueue - test_gdbm + test_dbm_gnu test_gzip test_openpty test_zipfile @@ -1015,7 +1015,7 @@ test_curses test_dl test_epoll - test_gdbm + test_dbm_gnu test_gzip test_largefile test_locale @@ -1031,7 +1031,7 @@ """ test_curses test_dl - test_gdbm + test_dbm_gnu test_epoll test_largefile test_locale @@ -1076,7 +1076,7 @@ test_bsddb test_bsddb3 test_epoll - test_gdbm + test_dbm_gnu test_locale test_ossaudiodev test_pep277 @@ -1093,7 +1093,7 @@ test_bz2 test_dl test_epoll - test_gdbm + test_dbm_gnu test_gzip test_kqueue test_ossaudiodev @@ -1108,7 +1108,7 @@ test_ctypes test_dl test_epoll - test_gdbm + test_dbm_gnu test_locale test_normalization test_ossaudiodev @@ -1123,7 +1123,7 @@ test_curses test_dl test_epoll - test_gdbm + test_dbm_gnu test_locale test_ossaudiodev test_pep277 From python-3000-checkins at python.org Mon May 26 21:41:10 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 21:41:10 +0200 (CEST) Subject: [Python-3000-checkins] r63720 - python/branches/py3k Message-ID: <20080526194110.CBE6F1E4019@bag.python.org> Author: benjamin.peterson Date: Mon May 26 21:41:10 2008 New Revision: 63720 Log: Blocked revisions 63719 via svnmerge ........ r63719 | benjamin.peterson | 2008-05-26 14:37:11 -0500 (Mon, 26 May 2008) | 2 lines wrap line ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Mon May 26 21:44:01 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 21:44:01 +0200 (CEST) Subject: [Python-3000-checkins] r63722 - python/branches/py3k Message-ID: <20080526194401.4D5F51E401E@bag.python.org> Author: benjamin.peterson Date: Mon May 26 21:44:01 2008 New Revision: 63722 Log: Blocked revisions 63721 via svnmerge ........ r63721 | benjamin.peterson | 2008-05-26 14:41:53 -0500 (Mon, 26 May 2008) | 2 lines warn about some members of the commands module ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Mon May 26 22:47:11 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Mon, 26 May 2008 22:47:11 +0200 (CEST) Subject: [Python-3000-checkins] r63727 - python/branches/py3k Message-ID: <20080526204711.D1F551E4008@bag.python.org> Author: benjamin.peterson Date: Mon May 26 22:47:11 2008 New Revision: 63727 Log: Blocked revisions 63725 via svnmerge ........ r63725 | benjamin.peterson | 2008-05-26 15:41:45 -0500 (Mon, 26 May 2008) | 2 lines take Brett's advice on a few warnings ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Mon May 26 23:41:43 2008 From: python-3000-checkins at python.org (amaury.forgeotdarc) Date: Mon, 26 May 2008 23:41:43 +0200 (CEST) Subject: [Python-3000-checkins] r63731 - in python/branches/py3k: Lib/test/test_marshal.py Misc/NEWS Python/marshal.c Message-ID: <20080526214143.0CADB1E4008@bag.python.org> Author: amaury.forgeotdarc Date: Mon May 26 23:41:42 2008 New Revision: 63731 Log: #2957: marshal recursion limit exceeded when importing a large .pyc file Modified: python/branches/py3k/Lib/test/test_marshal.py python/branches/py3k/Misc/NEWS python/branches/py3k/Python/marshal.c Modified: python/branches/py3k/Lib/test/test_marshal.py ============================================================================== --- python/branches/py3k/Lib/test/test_marshal.py (original) +++ python/branches/py3k/Lib/test/test_marshal.py Mon May 26 23:41:42 2008 @@ -113,6 +113,12 @@ new = marshal.loads(marshal.dumps(co)) self.assertEqual(co, new) + def test_many_codeobjects(self): + # Issue2957: bad recursion count on code objects + count = 5000 # more than MAX_MARSHAL_STACK_DEPTH + codes = (ExceptionTestCase.test_exceptions.__code__,) * count + marshal.loads(marshal.dumps(codes)) + class ContainerTestCase(unittest.TestCase, HelperMixin): d = {'astring': 'foo at bar.baz.spam', 'afloat': 7283.43, Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Mon May 26 23:41:42 2008 @@ -12,6 +12,10 @@ Core and Builtins ----------------- +- Issue #2957: Corrected a ValueError "recursion limit exceeded", when + unmarshalling many code objects, which happens when importing a + large .pyc file (~1000 functions). + - Issue #2963: fix merging oversight that disabled method cache for all types. Modified: python/branches/py3k/Python/marshal.c ============================================================================== --- python/branches/py3k/Python/marshal.c (original) +++ python/branches/py3k/Python/marshal.c Mon May 26 23:41:42 2008 @@ -913,8 +913,6 @@ Py_XDECREF(filename); Py_XDECREF(name); Py_XDECREF(lnotab); - - return v; } retval = v; break; From python-3000-checkins at python.org Tue May 27 02:36:20 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Tue, 27 May 2008 02:36:20 +0200 (CEST) Subject: [Python-3000-checkins] r63735 - python/branches/py3k/Objects/bytesobject.c Message-ID: <20080527003620.CF3871E400E@bag.python.org> Author: benjamin.peterson Date: Tue May 27 02:36:20 2008 New Revision: 63735 Log: clean up some docstrings and errors in bytesobject.c Modified: python/branches/py3k/Objects/bytesobject.c Modified: python/branches/py3k/Objects/bytesobject.c ============================================================================== --- python/branches/py3k/Objects/bytesobject.c (original) +++ python/branches/py3k/Objects/bytesobject.c Tue May 27 02:36:20 2008 @@ -1,7 +1,4 @@ -/* String object implementation */ - -/* XXX This is now called 'bytes' as far as the user is concerned. - Many docstrings and error messages need to be cleaned up. */ +/* bytes object implementation */ #define PY_SSIZE_T_CLEAN @@ -116,7 +113,7 @@ size = strlen(str); if (size > PY_SSIZE_T_MAX) { PyErr_SetString(PyExc_OverflowError, - "string is too long for a Python string"); + "byte string is too long"); return NULL; } if (size == 0 && (op = nullstring) != NULL) { @@ -735,7 +732,7 @@ size = Py_SIZE(a) * n; if (n && size / n != Py_SIZE(a)) { PyErr_SetString(PyExc_OverflowError, - "repeated string is too long"); + "repeated bytes are too long"); return NULL; } if (size == Py_SIZE(a) && PyBytes_CheckExact(a)) { @@ -745,7 +742,7 @@ nbytes = (size_t)size; if (nbytes + sizeof(PyBytesObject) <= nbytes) { PyErr_SetString(PyExc_OverflowError, - "repeated string is too long"); + "repeated bytes are too long"); return NULL; } op = (PyBytesObject *) @@ -799,7 +796,7 @@ string_item(PyBytesObject *a, register Py_ssize_t i) { if (i < 0 || i >= Py_SIZE(a)) { - PyErr_SetString(PyExc_IndexError, "string index out of range"); + PyErr_SetString(PyExc_IndexError, "index out of range"); return NULL; } return PyLong_FromLong((unsigned char)a->ob_sval[i]); @@ -908,7 +905,7 @@ i += PyBytes_GET_SIZE(self); if (i < 0 || i >= PyBytes_GET_SIZE(self)) { PyErr_SetString(PyExc_IndexError, - "string index out of range"); + "index out of range"); return NULL; } return PyLong_FromLong((unsigned char)self->ob_sval[i]); @@ -958,7 +955,7 @@ } else { PyErr_Format(PyExc_TypeError, - "string indices must be integers, not %.200s", + "byte indices must be integers, not %.200s", Py_TYPE(item)->tp_name); return NULL; } @@ -1348,7 +1345,7 @@ } PyDoc_STRVAR(rsplit__doc__, -"B.rsplit([sep[, maxsplit]]) -> list of strings\n\ +"B.rsplit([sep[, maxsplit]]) -> list of bytes\n\ \n\ Return a list of the sections in B, using sep as the delimiter,\n\ starting at the end of B and working to the front.\n\ @@ -1482,7 +1479,7 @@ sz += seplen; if (sz < old_sz || sz > PY_SSIZE_T_MAX) { PyErr_SetString(PyExc_OverflowError, - "join() result is too long for a Python string"); + "join() result is too long for bytes"); Py_DECREF(seq); return NULL; } @@ -2091,13 +2088,13 @@ product = count * to_len; if (product / to_len != count) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacement bytes are too long"); return NULL; } result_len = product + self_len; if (result_len < 0) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacement bytes are too long"); return NULL; } @@ -2346,13 +2343,13 @@ product = count * (to_len-1); if (product / (to_len-1) != count) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacement bytes are too long"); return NULL; } result_len = self_len + product; if (result_len < 0) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacment bytes are too long"); return NULL; } @@ -2416,13 +2413,13 @@ product = count * (to_len-from_len); if (product / (to_len-from_len) != count) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacement bytes are too long"); return NULL; } result_len = self_len + product; if (result_len < 0) { PyErr_SetString(PyExc_OverflowError, - "replace string is too long"); + "replacement bytes are too long"); return NULL; } @@ -2620,7 +2617,7 @@ Return True if B starts with the specified prefix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ With optional end, stop comparing B at that position.\n\ -prefix can also be a tuple of strings to try."); +prefix can also be a tuple of bytes to try."); static PyObject * string_startswith(PyBytesObject *self, PyObject *args) @@ -2661,7 +2658,7 @@ Return True if B ends with the specified suffix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ With optional end, stop comparing B at that position.\n\ -suffix can also be a tuple of strings to try."); +suffix can also be a tuple of bytes to try."); static PyObject * string_endswith(PyBytesObject *self, PyObject *args) @@ -3310,7 +3307,7 @@ nullstring = NULL; } -/*********************** Str Iterator ****************************/ +/*********************** Bytes Iterator ****************************/ typedef struct { PyObject_HEAD From python-3000-checkins at python.org Tue May 27 03:20:50 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Tue, 27 May 2008 03:20:50 +0200 (CEST) Subject: [Python-3000-checkins] r63737 - python/branches/py3k Message-ID: <20080527012050.2C09E1E4008@bag.python.org> Author: benjamin.peterson Date: Tue May 27 03:20:49 2008 New Revision: 63737 Log: Blocked revisions 63736 via svnmerge ........ r63736 | benjamin.peterson | 2008-05-26 20:18:39 -0500 (Mon, 26 May 2008) | 2 lines remove some __getslice__ ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Tue May 27 03:43:58 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Tue, 27 May 2008 03:43:58 +0200 (CEST) Subject: [Python-3000-checkins] r63739 - python/branches/py3k Message-ID: <20080527014358.48E771E4008@bag.python.org> Author: benjamin.peterson Date: Tue May 27 03:43:57 2008 New Revision: 63739 Log: Blocked revisions 63738 via svnmerge ........ r63738 | benjamin.peterson | 2008-05-26 20:42:29 -0500 (Mon, 26 May 2008) | 4 lines Improvements for test_py3kwarn - Always show warnings so they are always catchable - Make test_os_path_walk faster by walking a less populous directory ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Tue May 27 22:34:09 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Tue, 27 May 2008 22:34:09 +0200 (CEST) Subject: [Python-3000-checkins] r63753 - in python/branches/py3k: Demo/newmetaclasses/Eiffel.py Demo/newmetaclasses/Enum.py Lib/test/test_descr.py Parser/asdl.py Tools/framer/framer/bases.py Message-ID: <20080527203409.B765E1E4008@bag.python.org> Author: georg.brandl Date: Tue May 27 22:34:09 2008 New Revision: 63753 Log: Remove __metaclass__ remains. Modified: python/branches/py3k/Demo/newmetaclasses/Eiffel.py python/branches/py3k/Demo/newmetaclasses/Enum.py python/branches/py3k/Lib/test/test_descr.py python/branches/py3k/Parser/asdl.py python/branches/py3k/Tools/framer/framer/bases.py Modified: python/branches/py3k/Demo/newmetaclasses/Eiffel.py ============================================================================== --- python/branches/py3k/Demo/newmetaclasses/Eiffel.py (original) +++ python/branches/py3k/Demo/newmetaclasses/Eiffel.py Tue May 27 22:34:09 2008 @@ -85,8 +85,8 @@ make_eiffel_method = EiffelDescriptor def _test(metaclass): - class Eiffel: - __metaclass__ = metaclass + class Eiffel(metaclass=metaclass): + pass class Test(Eiffel): Modified: python/branches/py3k/Demo/newmetaclasses/Enum.py ============================================================================== --- python/branches/py3k/Demo/newmetaclasses/Enum.py (original) +++ python/branches/py3k/Demo/newmetaclasses/Enum.py Tue May 27 22:34:09 2008 @@ -82,11 +82,11 @@ def __str__(self): return "%s.%s" % (self.__classname, self.__enumname) -class Enum: - __metaclass__ = EnumMetaclass +class Enum(metaclass=EnumMetaclass): + pass -class FullEnum: - __metaclass__ = FullEnumMetaclass +class FullEnum(metaclass=FullEnumMetaclass): + pass def _test(): Modified: python/branches/py3k/Lib/test/test_descr.py ============================================================================== --- python/branches/py3k/Lib/test/test_descr.py (original) +++ python/branches/py3k/Lib/test/test_descr.py Tue May 27 22:34:09 2008 @@ -440,7 +440,7 @@ self.assertEqual(a[100:200], (100,200)) def test_metaclass(self): - # Testing __metaclass__... + # Testing metaclasses... class C(metaclass=type): def __init__(self): self.__state = 0 Modified: python/branches/py3k/Parser/asdl.py ============================================================================== --- python/branches/py3k/Parser/asdl.py (original) +++ python/branches/py3k/Parser/asdl.py Tue May 27 22:34:09 2008 @@ -10,8 +10,6 @@ Changes for Python: Add support for module versions """ -#__metaclass__ = type - import os import sys import traceback Modified: python/branches/py3k/Tools/framer/framer/bases.py ============================================================================== --- python/branches/py3k/Tools/framer/framer/bases.py (original) +++ python/branches/py3k/Tools/framer/framer/bases.py Tue May 27 22:34:09 2008 @@ -96,8 +96,8 @@ p("}") -class Module: - __metaclass__ = ModuleMetaclass +class Module(metaclass=ModuleMetaclass): + pass class TypeMetaclass(BaseMetaclass): @@ -211,5 +211,5 @@ p(template.type_init_type) p(template.module_add_type) -class Type: - __metaclass__ = TypeMetaclass +class Type(metaclass=TypeMetaclass): + pass From python-3000-checkins at python.org Wed May 28 17:56:30 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Wed, 28 May 2008 17:56:30 +0200 (CEST) Subject: [Python-3000-checkins] r63762 - in python/branches/py3k: Doc/library/http.cookies.rst Lib/http/cookies.py Lib/test/test_http_cookies.py Misc/NEWS Message-ID: <20080528155630.ED1621E4008@bag.python.org> Author: georg.brandl Date: Wed May 28 17:56:30 2008 New Revision: 63762 Log: Remove deprecated SmartCookie and SerialCookie classes. Modified: python/branches/py3k/Doc/library/http.cookies.rst python/branches/py3k/Lib/http/cookies.py python/branches/py3k/Lib/test/test_http_cookies.py python/branches/py3k/Misc/NEWS Modified: python/branches/py3k/Doc/library/http.cookies.rst ============================================================================== --- python/branches/py3k/Doc/library/http.cookies.rst (original) +++ python/branches/py3k/Doc/library/http.cookies.rst Wed May 28 17:56:30 2008 @@ -39,36 +39,6 @@ and :meth:`value_encode` to be the identity and :func:`str` respectively. -.. class:: SerialCookie([input]) - - This class derives from :class:`BaseCookie` and overrides :meth:`value_decode` - and :meth:`value_encode` to be the :func:`pickle.loads` and - :func:`pickle.dumps`. - - .. deprecated:: 2.3 - Reading pickled values from untrusted cookie data is a huge security hole, as - pickle strings can be crafted to cause arbitrary code to execute on your server. - It is supported for backwards compatibility only, and may eventually go away. - - -.. class:: SmartCookie([input]) - - This class derives from :class:`BaseCookie`. It overrides :meth:`value_decode` - to be :func:`pickle.loads` if it is a valid pickle, and otherwise the value - itself. It overrides :meth:`value_encode` to be :func:`pickle.dumps` unless it - is a string, in which case it returns the value itself. - - .. deprecated:: 2.3 - The same security warning from :class:`SerialCookie` applies here. - -A further security note is warranted. For backwards compatibility, the -:mod:`http.cookies` module exports a class named :class:`Cookie` which is just an -alias for :class:`SmartCookie`. This is probably a mistake and will likely be -removed in a future version. You should not use the :class:`Cookie` class in -your applications, for the same reason why you should not use the -:class:`SerialCookie` class. - - .. seealso:: Module :mod:`http.cookiejar` @@ -212,8 +182,6 @@ >>> from http import cookies >>> C = cookies.SimpleCookie() - >>> C = cookies.SerialCookie() - >>> C = cookies.SmartCookie() >>> C["fig"] = "newton" >>> C["sugar"] = "wafer" >>> print(C) # generate HTTP headers @@ -222,28 +190,28 @@ >>> print(C.output()) # same thing Set-Cookie: fig=newton Set-Cookie: sugar=wafer - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["rocky"] = "road" >>> C["rocky"]["path"] = "/cookie" >>> print(C.output(header="Cookie:")) Cookie: rocky=road; Path=/cookie >>> print(C.output(attrs=[], header="Cookie:")) Cookie: rocky=road - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C.load("chips=ahoy; vienna=finger") # load from a string (HTTP header) >>> print(C) Set-Cookie: chips=ahoy Set-Cookie: vienna=finger - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";') >>> print(C) Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;" - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["oreo"] = "doublestuff" >>> C["oreo"]["path"] = "/" >>> print(C) Set-Cookie: oreo=doublestuff; Path=/ - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["twix"] = "none for you" >>> C["twix"].value 'none for you' @@ -257,24 +225,3 @@ >>> print(C) Set-Cookie: number=7 Set-Cookie: string=seven - >>> C = cookies.SerialCookie() - >>> C["number"] = 7 - >>> C["string"] = "seven" - >>> C["number"].value - 7 - >>> C["string"].value - 'seven' - >>> print(C) - Set-Cookie: number="I7\012." - Set-Cookie: string="S'seven'\012p1\012." - >>> C = cookies.SmartCookie() - >>> C["number"] = 7 - >>> C["string"] = "seven" - >>> C["number"].value - 7 - >>> C["string"].value - 'seven' - >>> print(C) - Set-Cookie: number="I7\012." - Set-Cookie: string=seven - Modified: python/branches/py3k/Lib/http/cookies.py ============================================================================== --- python/branches/py3k/Lib/http/cookies.py (original) +++ python/branches/py3k/Lib/http/cookies.py Wed May 28 17:56:30 2008 @@ -50,23 +50,14 @@ >>> from http import cookies -Most of the time you start by creating a cookie. Cookies come in -three flavors, each with slightly different encoding semantics, but -more on that later. +Most of the time you start by creating a cookie. >>> C = cookies.SimpleCookie() - >>> C = cookies.SerialCookie() - >>> C = cookies.SmartCookie() - -[Note: Long-time users of cookies.py will remember using -cookies.Cookie() to create an Cookie object. Although deprecated, it -is still supported by the code. See the Backward Compatibility notes -for more information.] Once you've created your Cookie, you can add values just as if it were a dictionary. - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["fig"] = "newton" >>> C["sugar"] = "wafer" >>> C.output() @@ -77,7 +68,7 @@ default behavior. You can change the header and printed attributes by using the .output() function - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["rocky"] = "road" >>> C["rocky"]["path"] = "/cookie" >>> print(C.output(header="Cookie:")) @@ -89,7 +80,7 @@ CGI script, you would use this method to extract the cookies from the HTTP_COOKIE environment variable. - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C.load("chips=ahoy; vienna=finger") >>> C.output() 'Set-Cookie: chips=ahoy\r\nSet-Cookie: vienna=finger' @@ -98,7 +89,7 @@ within a string. Escaped quotation marks, nested semicolons, and other such trickeries do not confuse it. - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C.load('keebler="E=everybody; L=\\"Loves\\"; fudge=\\012;";') >>> print(C) Set-Cookie: keebler="E=everybody; L=\"Loves\"; fudge=\012;" @@ -107,7 +98,7 @@ Cookie attributes. Here's an example which sets the Path attribute. - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["oreo"] = "doublestuff" >>> C["oreo"]["path"] = "/" >>> print(C) @@ -116,21 +107,11 @@ Each dictionary element has a 'value' attribute, which gives you back the value associated with the key. - >>> C = cookies.SmartCookie() + >>> C = cookies.SimpleCookie() >>> C["twix"] = "none for you" >>> C["twix"].value 'none for you' - -A Bit More Advanced -------------------- - -As mentioned before, there are three different flavors of Cookie -objects, each with different encoding/decoding semantics. This -section briefly discusses the differences. - -SimpleCookie - The SimpleCookie expects that all values should be standard strings. Just to be sure, SimpleCookie invokes the str() builtin to convert the value to a string, when the values are set dictionary-style. @@ -145,62 +126,6 @@ >>> C.output() 'Set-Cookie: number=7\r\nSet-Cookie: string=seven' - -SerialCookie - -The SerialCookie expects that all values should be serialized using -pickle. As a result of serializing, SerialCookie can save almost any -Python object to a value, and recover the exact same object when the -cookie has been returned. (SerialCookie can yield some -strange-looking cookie values, however.) - - >>> C = cookies.SerialCookie() - >>> C["number"] = 7 - >>> C["string"] = "seven" - >>> C["number"].value - 7 - >>> C["string"].value - 'seven' - >>> C.output() - 'Set-Cookie: number="L7\\012."\r\nSet-Cookie: string="Vseven\\012p0\\012."' - -Be warned, however, if SerialCookie cannot de-serialize a value (because -it isn't a valid pickle'd object), IT WILL RAISE AN EXCEPTION. - - -SmartCookie - -The SmartCookie combines aspects of each of the other two flavors. -When setting a value in a dictionary-fashion, the SmartCookie will -serialize (ala pickle) the value *if and only if* it isn't a -Python string. String objects are *not* serialized. Similarly, -when the load() method parses out values, it attempts to de-serialize -the value. If it fails, then it fallsback to treating the value -as a string. - - >>> C = cookies.SmartCookie() - >>> C["number"] = 7 - >>> C["string"] = "seven" - >>> C["number"].value - 7 - >>> C["string"].value - 'seven' - >>> C.output() - 'Set-Cookie: number="L7\\012."\r\nSet-Cookie: string=seven' - - -Backwards Compatibility ------------------------ - -In order to keep compatibilty with earlier versions of Cookie.py, -it is still possible to use cookies.Cookie() to create a Cookie. In -fact, this simply returns a SmartCookie. - - >>> C = cookies.Cookie() - >>> print(C.__class__.__name__) - SmartCookie - - Finis. """ #" # ^ @@ -215,8 +140,7 @@ import re, warnings -__all__ = ["CookieError","BaseCookie","SimpleCookie","SerialCookie", - "SmartCookie","Cookie"] +__all__ = ["CookieError", "BaseCookie", "SimpleCookie"] _nulljoin = ''.join _semispacejoin = '; '.join @@ -653,70 +577,6 @@ return strval, _quote( strval ) # end SimpleCookie -class SerialCookie(BaseCookie): - """SerialCookie - SerialCookie supports arbitrary objects as cookie values. All - values are serialized (using pickle) before being sent to the - client. All incoming values are assumed to be valid Pickle - representations. IF AN INCOMING VALUE IS NOT IN A VALID PICKLE - FORMAT, THEN AN EXCEPTION WILL BE RAISED. - - Note: Large cookie values add overhead because they must be - retransmitted on every HTTP transaction. - - Note: HTTP has a 2k limit on the size of a cookie. This class - does not check for this limit, so be careful!!! - """ - def __init__(self, input=None): - warnings.warn("SerialCookie class is insecure; do not use it", - DeprecationWarning) - BaseCookie.__init__(self, input) - # end __init__ - def value_decode(self, val): - # This could raise an exception! - return loads( _unquote(val).encode('latin-1') ), val - def value_encode(self, val): - return val, _quote( dumps(val, 0).decode('latin-1') ) -# end SerialCookie - -class SmartCookie(BaseCookie): - """SmartCookie - SmartCookie supports arbitrary objects as cookie values. If the - object is a string, then it is quoted. If the object is not a - string, however, then SmartCookie will use pickle to serialize - the object into a string representation. - - Note: Large cookie values add overhead because they must be - retransmitted on every HTTP transaction. - - Note: HTTP has a 2k limit on the size of a cookie. This class - does not check for this limit, so be careful!!! - """ - def __init__(self, input=None): - warnings.warn("Cookie/SmartCookie class is insecure; do not use it", - DeprecationWarning) - BaseCookie.__init__(self, input) - # end __init__ - def value_decode(self, val): - strval = _unquote(val) - try: - return loads(strval.encode('latin-1')), val - except: - return strval, val - def value_encode(self, val): - if isinstance(val, str): - return val, _quote(val) - else: - return val, _quote( dumps(val, 0).decode('latin-1') ) -# end SmartCookie - - -########################################################### -# Backwards Compatibility: Don't break any existing code! - -# We provide Cookie() as an alias for SmartCookie() -Cookie = SmartCookie - # ########################################################### @@ -726,8 +586,3 @@ if __name__ == "__main__": _test() - - -#Local Variables: -#tab-width: 4 -#end: Modified: python/branches/py3k/Lib/test/test_http_cookies.py ============================================================================== --- python/branches/py3k/Lib/test/test_http_cookies.py (original) +++ python/branches/py3k/Lib/test/test_http_cookies.py Wed May 28 17:56:30 2008 @@ -10,7 +10,6 @@ DeprecationWarning) class CookieTests(unittest.TestCase): - # Currently this only tests SimpleCookie def test_basic(self): cases = [ { 'data': 'chips=ahoy; vienna=finger', Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Wed May 28 17:56:30 2008 @@ -60,6 +60,9 @@ Library ------- +- The deprecated ``SmartCookie`` and ``SimpleCookie`` classes have + been removed from ``http.cookies``. + - The ``commands`` module has been removed. Its getoutput() and getstatusoutput() functions have been moved to the ``subprocess`` module. From python-3000-checkins at python.org Thu May 29 07:09:29 2008 From: python-3000-checkins at python.org (brett.cannon) Date: Thu, 29 May 2008 07:09:29 +0200 (CEST) Subject: [Python-3000-checkins] r63768 - python/branches/py3k Message-ID: <20080529050929.2F5671E4003@bag.python.org> Author: brett.cannon Date: Thu May 29 07:09:28 2008 New Revision: 63768 Log: Blocked revisions 63767 via svnmerge ........ r63767 | brett.cannon | 2008-05-28 22:08:50 -0700 (Wed, 28 May 2008) | 4 lines UserString.MutableString has been removed in Python 3.0. Works on issue #2877. Thanks Quentin Gallet-Gilles for the patch. ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Thu May 29 09:19:00 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Thu, 29 May 2008 09:19:00 +0200 (CEST) Subject: [Python-3000-checkins] r63777 - python/branches/py3k/Lib/tkinter/__init__.py Message-ID: <20080529071900.7F5261E4005@bag.python.org> Author: georg.brandl Date: Thu May 29 09:19:00 2008 New Revision: 63777 Log: #2906: accept lists for options, and some cosmetic fixes in Tkinter. Modified: python/branches/py3k/Lib/tkinter/__init__.py Modified: python/branches/py3k/Lib/tkinter/__init__.py ============================================================================== --- python/branches/py3k/Lib/tkinter/__init__.py (original) +++ python/branches/py3k/Lib/tkinter/__init__.py Thu May 29 09:19:00 2008 @@ -1051,6 +1051,12 @@ if k[-1] == '_': k = k[:-1] if hasattr(v, '__call__'): v = self._register(v) + elif isinstance(v, (tuple, list)): + for item in v: + if not isinstance(item, (basestring, int)): + break + else: + v = ' '.join(map(str, v)) res = res + ('-'+k, v) return res def nametowidget(self, name): @@ -1090,7 +1096,6 @@ if self._tclCommands is None: self._tclCommands = [] self._tclCommands.append(name) - #print '+ Tkinter created command', name return name register = _register def _root(self): @@ -1740,10 +1745,11 @@ after=widget - pack it after you have packed widget anchor=NSEW (or subset) - position widget according to given direction - before=widget - pack it before you will pack widget + before=widget - pack it before you will pack widget expand=bool - expand widget if parent size grows fill=NONE or X or Y or BOTH - fill widget if widget grows in=master - use master to contain this widget + in_=master - see 'in' option description ipadx=amount - add internal padding in x direction ipady=amount - add internal padding in y direction padx=amount - add padding in x direction @@ -1781,29 +1787,26 @@ Base class to use the methods place_* in every widget.""" def place_configure(self, cnf={}, **kw): """Place a widget in the parent widget. Use as options: - in=master - master relative to which the widget is placed. + in=master - master relative to which the widget is placed + in_=master - see 'in' option description x=amount - locate anchor of this widget at position x of master y=amount - locate anchor of this widget at position y of master relx=amount - locate anchor of this widget between 0.0 and 1.0 relative to width of master (1.0 is right edge) - rely=amount - locate anchor of this widget between 0.0 and 1.0 + rely=amount - locate anchor of this widget between 0.0 and 1.0 relative to height of master (1.0 is bottom edge) - anchor=NSEW (or subset) - position anchor according to given direction + anchor=NSEW (or subset) - position anchor according to given direction width=amount - width of this widget in pixel height=amount - height of this widget in pixel relwidth=amount - width of this widget between 0.0 and 1.0 relative to width of master (1.0 is the same width - as the master) - relheight=amount - height of this widget between 0.0 and 1.0 + as the master) + relheight=amount - height of this widget between 0.0 and 1.0 relative to height of master (1.0 is the same - height as the master) - bordermode="inside" or "outside" - whether to take border width of master widget - into account - """ - for k in ['in_']: - if k in kw: - kw[k[:-1]] = kw[k] - del kw[k] + height as the master) + bordermode="inside" or "outside" - whether to take border width of + master widget into account + """ self.tk.call( ('place', 'configure', self._w) + self._options(cnf, kw)) @@ -1838,6 +1841,7 @@ column=number - use cell identified with given column (starting with 0) columnspan=number - this widget will span several columns in=master - use master to contain this widget + in_=master - see 'in' option description ipadx=amount - add internal padding in x direction ipady=amount - add internal padding in y direction padx=amount - add padding in x direction From python-3000-checkins at python.org Thu May 29 09:20:20 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Thu, 29 May 2008 09:20:20 +0200 (CEST) Subject: [Python-3000-checkins] r63779 - python/branches/py3k Message-ID: <20080529072020.AD7131E4003@bag.python.org> Author: georg.brandl Date: Thu May 29 09:20:20 2008 New Revision: 63779 Log: Blocked revisions 63776 via svnmerge ........ r63776 | georg.brandl | 2008-05-29 09:18:49 +0200 (Thu, 29 May 2008) | 2 lines #2906: accept lists for options, and some cosmetic fixes in Tkinter. ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Thu May 29 21:54:56 2008 From: python-3000-checkins at python.org (thomas.heller) Date: Thu, 29 May 2008 21:54:56 +0200 (CEST) Subject: [Python-3000-checkins] r63793 - in python/branches/py3k: Lib/ctypes/test/test_pointers.py Modules/_ctypes/_ctypes.c Message-ID: <20080529195456.6A0441E4003@bag.python.org> Author: thomas.heller Date: Thu May 29 21:54:39 2008 New Revision: 63793 Log: Merged revisions 63791-63792 via svnmerge from svn+ssh://pythondev at svn.python.org/python/trunk ........ r63791 | thomas.heller | 2008-05-29 21:18:12 +0200 (Do, 29 Mai 2008) | 1 line Fix compiler warning. ........ r63792 | thomas.heller | 2008-05-29 21:42:34 +0200 (Do, 29 Mai 2008) | 1 line ctypes NULL function pointers have a boolean False value now. ........ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Lib/ctypes/test/test_pointers.py python/branches/py3k/Modules/_ctypes/_ctypes.c Modified: python/branches/py3k/Lib/ctypes/test/test_pointers.py ============================================================================== --- python/branches/py3k/Lib/ctypes/test/test_pointers.py (original) +++ python/branches/py3k/Lib/ctypes/test/test_pointers.py Thu May 29 21:54:39 2008 @@ -175,5 +175,13 @@ self.assertRaises(TypeError, c_void_p, 3.14) # make sure floats are NOT accepted self.assertRaises(TypeError, c_void_p, object()) # nor other objects + def test_pointers_bool(self): + # NULL pointers have a boolean False value, non-NULL pointers True. + self.failUnlessEqual(bool(POINTER(c_int)()), False) + self.failUnlessEqual(bool(pointer(c_int())), True) + + self.failUnlessEqual(bool(CFUNCTYPE(None)(0)), False) + self.failUnlessEqual(bool(CFUNCTYPE(None)(42)), True) + if __name__ == '__main__': unittest.main() Modified: python/branches/py3k/Modules/_ctypes/_ctypes.c ============================================================================== --- python/branches/py3k/Modules/_ctypes/_ctypes.c (original) +++ python/branches/py3k/Modules/_ctypes/_ctypes.c Thu May 29 21:54:39 2008 @@ -474,7 +474,7 @@ static PyObject * CDataType_from_buffer_copy(PyObject *type, PyObject *args) { - void *buffer; + const void *buffer; Py_ssize_t buffer_len; Py_ssize_t offset = 0; PyObject *obj, *result; @@ -3849,6 +3849,25 @@ self); } +static int +Pointer_bool(CDataObject *self) +{ + return *(void **)self->b_ptr != NULL; +} + +static PyNumberMethods Pointer_as_number = { + 0, /* nb_add */ + 0, /* nb_subtract */ + 0, /* nb_multiply */ + 0, /* nb_remainder */ + 0, /* nb_divmod */ + 0, /* nb_power */ + 0, /* nb_negative */ + 0, /* nb_positive */ + 0, /* nb_absolute */ + (inquiry)Pointer_bool, /* nb_bool */ +}; + PyTypeObject CFuncPtr_Type = { PyVarObject_HEAD_INIT(NULL, 0) "_ctypes.CFuncPtr", @@ -3860,7 +3879,7 @@ 0, /* tp_setattr */ 0, /* tp_compare */ (reprfunc)CFuncPtr_repr, /* tp_repr */ - 0, /* tp_as_number */ + &Pointer_as_number, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ 0, /* tp_hash */ @@ -4933,25 +4952,6 @@ Pointer_subscript, }; -static int -Pointer_bool(CDataObject *self) -{ - return *(void **)self->b_ptr != NULL; -} - -static PyNumberMethods Pointer_as_number = { - 0, /* nb_add */ - 0, /* nb_subtract */ - 0, /* nb_multiply */ - 0, /* nb_remainder */ - 0, /* nb_divmod */ - 0, /* nb_power */ - 0, /* nb_negative */ - 0, /* nb_positive */ - 0, /* nb_absolute */ - (inquiry)Pointer_bool, /* nb_bool */ -}; - PyTypeObject Pointer_Type = { PyVarObject_HEAD_INIT(NULL, 0) "_ctypes._Pointer", From python-3000-checkins at python.org Thu May 29 23:09:51 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Thu, 29 May 2008 23:09:51 +0200 (CEST) Subject: [Python-3000-checkins] r63795 - in python/branches/py3k: Lib/platform.py Modules/_gestalt.c setup.py Message-ID: <20080529210951.D3C9F1E4016@bag.python.org> Author: benjamin.peterson Date: Thu May 29 23:09:51 2008 New Revision: 63795 Log: add the gestalt module back as _gestalt Added: python/branches/py3k/Modules/_gestalt.c Modified: python/branches/py3k/Lib/platform.py python/branches/py3k/setup.py Modified: python/branches/py3k/Lib/platform.py ============================================================================== --- python/branches/py3k/Lib/platform.py (original) +++ python/branches/py3k/Lib/platform.py Thu May 29 23:09:51 2008 @@ -676,14 +676,13 @@ def _mac_ver_lookup(selectors,default=None): - from gestalt import gestalt - import MacOS + from _gestalt import gestalt l = [] append = l.append for selector in selectors: try: append(gestalt(selector)) - except (RuntimeError, MacOS.Error): + except (RuntimeError, OSError): append(default) return l @@ -709,8 +708,7 @@ """ # Check whether the version info module is available try: - import gestalt - import MacOS + import _gestalt except ImportError: return release,versioninfo,machine # Get the infos Added: python/branches/py3k/Modules/_gestalt.c ============================================================================== --- (empty file) +++ python/branches/py3k/Modules/_gestalt.c Thu May 29 23:09:51 2008 @@ -0,0 +1,72 @@ +/*********************************************************** +Copyright 1991-1997 by Stichting Mathematisch Centrum, Amsterdam, +The Netherlands. + + All Rights Reserved + +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee is hereby granted, +provided that the above copyright notice appear in all copies and that +both that copyright notice and this permission notice appear in +supporting documentation, and that the names of Stichting Mathematisch +Centrum or CWI not be used in advertising or publicity pertaining to +distribution of the software without specific, written prior permission. + +STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO +THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND +FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE +FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT +OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +******************************************************************/ + +/* Macintosh Gestalt interface */ + +#include "Python.h" + +#include + +/* Convert a 4-char string object argument to an OSType value */ +static int +convert_to_OSType(PyObject *v, OSType *pr) +{ + uint32_t tmp; + if (!PyUnicode_Check(v) || PyUnicode_GetSize(v) != 4) { + PyErr_SetString(PyExc_TypeError, + "OSType arg must be string of 4 chars"); + return 0; + } + memcpy((char *)&tmp, PyUnicode_AsString(v), 4); + *pr = (OSType)ntohl(tmp); + return 1; +} + +static PyObject * +gestalt_gestalt(PyObject *self, PyObject *args) +{ + OSErr iErr; + OSType selector; + SInt32 response; + if (!PyArg_ParseTuple(args, "O&", convert_to_OSType, &selector)) + return NULL; + iErr = Gestalt(selector, &response); + if (iErr != 0) { + PyErr_SetString(PyExc_OSError, + "non-zero exit code!"); + return NULL; + } + return PyLong_FromLong(response); +} + +static struct PyMethodDef gestalt_methods[] = { + {"gestalt", gestalt_gestalt, METH_VARARGS}, + {NULL, NULL} /* Sentinel */ +}; + +void +init_gestalt(void) +{ + Py_InitModule("_gestalt", gestalt_methods); +} Modified: python/branches/py3k/setup.py ============================================================================== --- python/branches/py3k/setup.py (original) +++ python/branches/py3k/setup.py Thu May 29 23:09:51 2008 @@ -1118,6 +1118,14 @@ else: missing.append('ossaudiodev') + if sys.platform == 'darwin': + exts.append( + Extension('_gestalt', ['_gestalt.c'], + extra_link_args=['-framework', 'Carbon']) + ) + else: + missing.append('_gestalt') + self.extensions.extend(exts) # Call the method for detecting whether _tkinter can be compiled From python-3000-checkins at python.org Thu May 29 23:12:04 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Thu, 29 May 2008 23:12:04 +0200 (CEST) Subject: [Python-3000-checkins] r63796 - python/branches/py3k Message-ID: <20080529211204.F01401E400F@bag.python.org> Author: benjamin.peterson Date: Thu May 29 23:12:04 2008 New Revision: 63796 Log: Unblocked revisions 63460,63464 via svnmerge ........ r63460 | ronald.oussoren | 2008-05-18 15:54:47 -0500 (Sun, 18 May 2008) | 6 lines - Add unittests for platform.mac_ver (or rather, ensure that the unittest for that function actually tests something on OSX). - Add documentation to platform.mac_ver that explains why the middle element of the return value will not contain useful information. ........ r63464 | benjamin.peterson | 2008-05-18 17:07:42 -0500 (Sun, 18 May 2008) | 2 lines fix test_platform (os was not imported) ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Thu May 29 23:22:40 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Thu, 29 May 2008 23:22:40 +0200 (CEST) Subject: [Python-3000-checkins] r63798 - in python/branches/py3k: Lib/platform.py Lib/test/test_platform.py Message-ID: <20080529212240.B06F61E4003@bag.python.org> Author: benjamin.peterson Date: Thu May 29 23:22:40 2008 New Revision: 63798 Log: Merged revisions 63460,63464 via svnmerge from svn+ssh://pythondev at svn.python.org/python/trunk ........ r63460 | ronald.oussoren | 2008-05-18 15:54:47 -0500 (Sun, 18 May 2008) | 6 lines - Add unittests for platform.mac_ver (or rather, ensure that the unittest for that function actually tests something on OSX). - Add documentation to platform.mac_ver that explains why the middle element of the return value will not contain useful information. ........ r63464 | benjamin.peterson | 2008-05-18 17:07:42 -0500 (Sun, 18 May 2008) | 2 lines fix test_platform (os was not imported) ........ Modified: python/branches/py3k/ (props changed) python/branches/py3k/Lib/platform.py python/branches/py3k/Lib/test/test_platform.py Modified: python/branches/py3k/Lib/platform.py ============================================================================== --- python/branches/py3k/Lib/platform.py (original) +++ python/branches/py3k/Lib/platform.py Thu May 29 23:22:40 2008 @@ -729,7 +729,11 @@ release = '%i.%i.%i' %(major, minor, patch) else: release = '%s.%i.%i' % (_bcd2str(major),minor,patch) + if sysu: + # NOTE: this block is left as documentation of the + # intention of this function, the 'sysu' gestalt is no + # longer available and there are no alternatives. major = int((sysu & 0xFF000000) >> 24) minor = (sysu & 0x00F00000) >> 20 bugfix = (sysu & 0x000F0000) >> 16 @@ -742,6 +746,8 @@ 0x60:'beta', 0x80:'final'}.get(stage,'') versioninfo = (version,stage,nonrel) + + if sysa: machine = {0x1: '68k', 0x2: 'PowerPC', Modified: python/branches/py3k/Lib/test/test_platform.py ============================================================================== --- python/branches/py3k/Lib/test/test_platform.py (original) +++ python/branches/py3k/Lib/test/test_platform.py Thu May 29 23:22:40 2008 @@ -1,4 +1,5 @@ import sys +import os import unittest import platform @@ -63,12 +64,29 @@ def test_mac_ver(self): res = platform.mac_ver() - try: - import gestalt - except ImportError: pass - else: - if sys.platform == 'darwin': - self.assert_(all(res)) + + if os.uname()[0] == 'Darwin': + # We're on a MacOSX system, check that + # the right version information is returned + fd = os.popen('sw_vers', 'r') + real_ver = None + for ln in fd: + if ln.startswith('ProductVersion:'): + real_ver = ln.strip().split()[-1] + break + fd.close() + self.failIf(real_ver is None) + self.assertEquals(res[0], real_ver) + + # res[1] claims to contain + # (version, dev_stage, non_release_version) + # That information is no longer available + self.assertEquals(res[1], ('', '', '')) + + if sys.byteorder == 'little': + self.assertEquals(res[2], 'i386') + else: + self.assertEquals(res[2], 'PowerPC') def test_dist(self): res = platform.dist() From python-3000-checkins at python.org Thu May 29 23:30:28 2008 From: python-3000-checkins at python.org (brett.cannon) Date: Thu, 29 May 2008 23:30:28 +0200 (CEST) Subject: [Python-3000-checkins] r63801 - python/branches/py3k Message-ID: <20080529213028.09D4F1E4003@bag.python.org> Author: brett.cannon Date: Thu May 29 23:30:27 2008 New Revision: 63801 Log: Blocked revisions 63800 via svnmerge ........ r63800 | brett.cannon | 2008-05-29 14:28:55 -0700 (Thu, 29 May 2008) | 1 line Note that UserList and UserString were moved to 'collections' in 3.0. ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Fri May 30 08:27:12 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Fri, 30 May 2008 08:27:12 +0200 (CEST) Subject: [Python-3000-checkins] r63803 - in python/branches/py3k/Doc/library: io.rst stringio.rst strings.rst Message-ID: <20080530062712.082451E4002@bag.python.org> Author: georg.brandl Date: Fri May 30 08:27:09 2008 New Revision: 63803 Log: #3007: remove stringio docs and fix a few nits in io docs. Removed: python/branches/py3k/Doc/library/stringio.rst Modified: python/branches/py3k/Doc/library/io.rst python/branches/py3k/Doc/library/strings.rst Modified: python/branches/py3k/Doc/library/io.rst ============================================================================== --- python/branches/py3k/Doc/library/io.rst (original) +++ python/branches/py3k/Doc/library/io.rst Fri May 30 08:27:09 2008 @@ -630,9 +630,9 @@ .. class:: StringIO([initial_value[, encoding[, errors[, newline]]]]) - An in-memory stream for text. It in inherits :class:`TextIOWrapper`. + An in-memory stream for text. It inherits :class:`TextIOWrapper`. - Create a new StringIO stream with an inital value, encoding, error handling, + Create a new StringIO stream with an initial value, encoding, error handling, and newline setting. See :class:`TextIOWrapper`\'s constructor for more information. @@ -641,8 +641,25 @@ .. method:: getvalue() - Return a ``str`` containing the entire contents of the buffer. + Return a ``str`` containing the entire contents of the buffer at any + time before the :class:`StringIO` object's :meth:`close` method is + called. + Example usage:: + + import io + + output = io.StringIO() + output.write('First line.\n') + print('Second line.', file=output) + + # Retrieve file contents -- this will be + # 'First line.\nSecond line.\n' + contents = output.getvalue() + + # Close object and discard memory buffer -- + # .getvalue() will now raise an exception. + output.close() .. class:: IncrementalNewlineDecoder Deleted: python/branches/py3k/Doc/library/stringio.rst ============================================================================== --- python/branches/py3k/Doc/library/stringio.rst Fri May 30 08:27:09 2008 +++ (empty file) @@ -1,117 +0,0 @@ -.. XXX this whole file is outdated - -:mod:`StringIO` --- Read and write strings as files -=================================================== - -.. module:: StringIO - :synopsis: Read and write strings as if they were files. - - -This module implements a file-like class, :class:`StringIO`, that reads and -writes a string buffer (also known as *memory files*). See the description of -file objects for operations (section :ref:`bltin-file-objects`). (For -standard strings, see :class:`str`.) - - -.. class:: StringIO([buffer]) - - When a :class:`StringIO` object is created, it can be initialized to an existing - string by passing the string to the constructor. If no string is given, the - :class:`StringIO` will start empty. In both cases, the initial file position - starts at zero. - -The following methods of :class:`StringIO` objects require special mention: - - -.. method:: StringIO.getvalue() - - Retrieve the entire contents of the "file" at any time before the - :class:`StringIO` object's :meth:`close` method is called. - - -.. method:: StringIO.close() - - Free the memory buffer. - -Example usage:: - - import StringIO - - output = StringIO.StringIO() - output.write('First line.\n') - print('Second line.', file=output) - - # Retrieve file contents -- this will be - # 'First line.\nSecond line.\n' - contents = output.getvalue() - - # Close object and discard memory buffer -- - # .getvalue() will now raise an exception. - output.close() - - -:mod:`cStringIO` --- Faster version of :mod:`StringIO` -====================================================== - -.. module:: cStringIO - :synopsis: Faster version of StringIO, but not subclassable. -.. moduleauthor:: Jim Fulton -.. sectionauthor:: Fred L. Drake, Jr. - - -The module :mod:`cStringIO` provides an interface similar to that of the -:mod:`StringIO` module. Heavy use of :class:`StringIO.StringIO` objects can be -made more efficient by using the function :func:`StringIO` from this module -instead. - -Since this module provides a factory function which returns objects of built-in -types, there's no way to build your own version using subclassing. Use the -original :mod:`StringIO` module in that case. - -Unlike the memory files implemented by the :mod:`StringIO` module, those -provided by this module are not able to accept strings that cannot be -encoded in plain ASCII. - -Calling :func:`StringIO` with a string parameter populates -the object with the buffer representation of the string, instead of -encoding the string. - -Another difference from the :mod:`StringIO` module is that calling -:func:`StringIO` with a string parameter creates a read-only object. Unlike an -object created without a string parameter, it does not have write methods. -These objects are not generally visible. They turn up in tracebacks as -:class:`StringI` and :class:`StringO`. - -The following data objects are provided as well: - - -.. data:: InputType - - The type object of the objects created by calling :func:`StringIO` with a string - parameter. - - -.. data:: OutputType - - The type object of the objects returned by calling :func:`StringIO` with no - parameters. - -There is a C API to the module as well; refer to the module source for more -information. - -Example usage:: - - import cStringIO - - output = cStringIO.StringIO() - output.write('First line.\n') - print('Second line.', file=output) - - # Retrieve file contents -- this will be - # 'First line.\nSecond line.\n' - contents = output.getvalue() - - # Close object and discard memory buffer -- - # .getvalue() will now raise an exception. - output.close() - Modified: python/branches/py3k/Doc/library/strings.rst ============================================================================== --- python/branches/py3k/Doc/library/strings.rst (original) +++ python/branches/py3k/Doc/library/strings.rst Fri May 30 08:27:09 2008 @@ -21,7 +21,6 @@ re.rst struct.rst difflib.rst - stringio.rst textwrap.rst codecs.rst unicodedata.rst From python-3000-checkins at python.org Fri May 30 08:27:55 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Fri, 30 May 2008 08:27:55 +0200 (CEST) Subject: [Python-3000-checkins] r63804 - python/branches/py3k/Lib/tkinter/__init__.py Message-ID: <20080530062755.093731E4002@bag.python.org> Author: georg.brandl Date: Fri May 30 08:27:54 2008 New Revision: 63804 Log: Remove basestring usage. Modified: python/branches/py3k/Lib/tkinter/__init__.py Modified: python/branches/py3k/Lib/tkinter/__init__.py ============================================================================== --- python/branches/py3k/Lib/tkinter/__init__.py (original) +++ python/branches/py3k/Lib/tkinter/__init__.py Fri May 30 08:27:54 2008 @@ -1053,7 +1053,7 @@ v = self._register(v) elif isinstance(v, (tuple, list)): for item in v: - if not isinstance(item, (basestring, int)): + if not isinstance(item, (str, int)): break else: v = ' '.join(map(str, v)) From python-3000-checkins at python.org Fri May 30 10:20:10 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Fri, 30 May 2008 10:20:10 +0200 (CEST) Subject: [Python-3000-checkins] r63809 - in python/branches/py3k/Objects: bytearrayobject.c bytesobject.c unicodeobject.c Message-ID: <20080530082010.C439C1E4017@bag.python.org> Author: georg.brandl Date: Fri May 30 10:20:09 2008 New Revision: 63809 Log: Fix all return types for str/bytes/bytearray docstrings and make the wording more consistent. Modified: python/branches/py3k/Objects/bytearrayobject.c python/branches/py3k/Objects/bytesobject.c python/branches/py3k/Objects/unicodeobject.c Modified: python/branches/py3k/Objects/bytearrayobject.c ============================================================================== --- python/branches/py3k/Objects/bytearrayobject.c (original) +++ python/branches/py3k/Objects/bytearrayobject.c Fri May 30 10:20:09 2008 @@ -1081,7 +1081,7 @@ } PyDoc_STRVAR(find__doc__, -"B.find(sub [,start [,end]]) -> int\n\ +"B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in B where subsection sub is found,\n\ such that sub is contained within s[start,end]. Optional\n\ @@ -1099,7 +1099,7 @@ } PyDoc_STRVAR(count__doc__, -"B.count(sub [,start [,end]]) -> int\n\ +"B.count(sub[, start[, end]]) -> int\n\ \n\ Return the number of non-overlapping occurrences of subsection sub in\n\ bytes B[start:end]. Optional arguments start and end are interpreted\n\ @@ -1132,7 +1132,7 @@ PyDoc_STRVAR(index__doc__, -"B.index(sub [,start [,end]]) -> int\n\ +"B.index(sub[, start[, end]]) -> int\n\ \n\ Like B.find() but raise ValueError when the subsection is not found."); @@ -1152,7 +1152,7 @@ PyDoc_STRVAR(rfind__doc__, -"B.rfind(sub [,start [,end]]) -> int\n\ +"B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where subsection sub is found,\n\ such that sub is contained within s[start,end]. Optional\n\ @@ -1171,7 +1171,7 @@ PyDoc_STRVAR(rindex__doc__, -"B.rindex(sub [,start [,end]]) -> int\n\ +"B.rindex(sub[, start[, end]]) -> int\n\ \n\ Like B.rfind() but raise ValueError when the subsection is not found."); @@ -1258,7 +1258,7 @@ PyDoc_STRVAR(startswith__doc__, -"B.startswith(prefix [,start [,end]]) -> bool\n\ +"B.startswith(prefix[, start[, end]]) -> bool\n\ \n\ Return True if B starts with the specified prefix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ @@ -1298,7 +1298,7 @@ } PyDoc_STRVAR(endswith__doc__, -"B.endswith(suffix [,start [,end]]) -> bool\n\ +"B.endswith(suffix[, start[, end]]) -> bool\n\ \n\ Return True if B ends with the specified suffix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ @@ -2001,7 +2001,7 @@ } if (to_len == 0) { - /* delete all occurances of 'from' bytes */ + /* delete all occurrences of 'from' bytes */ if (from_len == 1) { return replace_delete_single_character( self, from_s[0], maxcount); @@ -2037,7 +2037,7 @@ PyDoc_STRVAR(replace__doc__, -"B.replace(old, new[, count]) -> bytes\n\ +"B.replace(old, new[, count]) -> bytearray\n\ \n\ Return a copy of B with all occurrences of subsection\n\ old replaced by new. If the optional argument count is\n\ @@ -2187,7 +2187,7 @@ } PyDoc_STRVAR(split__doc__, -"B.split([sep[, maxsplit]]) -> list of bytearray\n\ +"B.split([sep[, maxsplit]]) -> list of bytearrays\n\ \n\ Return a list of the sections in B, using sep as the delimiter.\n\ If sep is not given, B is split on ASCII whitespace characters\n\ @@ -2294,7 +2294,7 @@ PyDoc_STRVAR(partition__doc__, "B.partition(sep) -> (head, sep, tail)\n\ \n\ -Searches for the separator sep in B, and returns the part before it,\n\ +Search for the separator sep in B, and return the part before it,\n\ the separator itself, and the part after it. If the separator is not\n\ found, returns B and two empty bytearray objects."); @@ -2321,8 +2321,8 @@ PyDoc_STRVAR(rpartition__doc__, "B.rpartition(sep) -> (tail, sep, head)\n\ \n\ -Searches for the separator sep in B, starting at the end of B,\n\ -and returns the part before it, the separator itself, and the\n\ +Search for the separator sep in B, starting at the end of B,\n\ +and return the part before it, the separator itself, and the\n\ part after it. If the separator is not found, returns two empty\n\ bytearray objects and B."); @@ -2421,7 +2421,7 @@ } PyDoc_STRVAR(rsplit__doc__, -"B.rsplit(sep[, maxsplit]) -> list of bytearray\n\ +"B.rsplit(sep[, maxsplit]) -> list of bytearrays\n\ \n\ Return a list of the sections in B, using sep as the delimiter,\n\ starting at the end of B and working to the front.\n\ @@ -2578,7 +2578,7 @@ } PyDoc_STRVAR(extend__doc__, -"B.extend(iterable int) -> None\n\ +"B.extend(iterable_of_ints) -> None\n\ \n\ Append all the elements from the iterator or sequence to the\n\ end of B."); @@ -2684,7 +2684,7 @@ PyDoc_STRVAR(remove__doc__, "B.remove(int) -> None\n\ \n\ -Remove the first occurance of a value in B."); +Remove the first occurrence of a value in B."); static PyObject * bytes_remove(PyByteArrayObject *self, PyObject *arg) { @@ -2735,7 +2735,8 @@ PyDoc_STRVAR(strip__doc__, "B.strip([bytes]) -> bytearray\n\ \n\ -Strip leading and trailing bytes contained in the argument.\n\ +Strip leading and trailing bytes contained in the argument\n\ +and return the result as a new bytearray.\n\ If the argument is omitted, strip ASCII whitespace."); static PyObject * bytes_strip(PyByteArrayObject *self, PyObject *args) @@ -2771,7 +2772,8 @@ PyDoc_STRVAR(lstrip__doc__, "B.lstrip([bytes]) -> bytearray\n\ \n\ -Strip leading bytes contained in the argument.\n\ +Strip leading bytes contained in the argument\n\ +and return the result as a new bytearray.\n\ If the argument is omitted, strip leading ASCII whitespace."); static PyObject * bytes_lstrip(PyByteArrayObject *self, PyObject *args) @@ -2804,7 +2806,8 @@ PyDoc_STRVAR(rstrip__doc__, "B.rstrip([bytes]) -> bytearray\n\ \n\ -Strip trailing bytes contained in the argument.\n\ +Strip trailing bytes contained in the argument\n\ +and return the result as a new bytearray.\n\ If the argument is omitted, strip trailing ASCII whitespace."); static PyObject * bytes_rstrip(PyByteArrayObject *self, PyObject *args) @@ -2835,9 +2838,9 @@ } PyDoc_STRVAR(decode_doc, -"B.decode([encoding[, errors]]) -> unicode object.\n\ +"B.decode([encoding[, errors]]) -> str\n\ \n\ -Decodes B using the codec registered for encoding. encoding defaults\n\ +Decode B using the codec registered for encoding. encoding defaults\n\ to the default encoding. errors may be given to set a different error\n\ handling scheme. Default is 'strict' meaning that encoding errors raise\n\ a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\ @@ -2860,7 +2863,7 @@ PyDoc_STRVAR(alloc_doc, "B.__alloc__() -> int\n\ \n\ -Returns the number of bytes actually allocated."); +Return the number of bytes actually allocated."); static PyObject * bytes_alloc(PyByteArrayObject *self) @@ -2869,9 +2872,10 @@ } PyDoc_STRVAR(join_doc, -"B.join(iterable_of_bytes) -> bytes\n\ +"B.join(iterable_of_bytes) -> bytearray\n\ \n\ -Concatenates any number of bytearray objects, with B in between each pair."); +Concatenate any number of bytes/bytearray objects, with B\n\ +in between each pair, and return the result as a new bytearray."); static PyObject * bytes_join(PyByteArrayObject *self, PyObject *it) @@ -2944,7 +2948,7 @@ } PyDoc_STRVAR(fromhex_doc, -"bytearray.fromhex(string) -> bytearray\n\ +"bytearray.fromhex(string) -> bytearray (static method)\n\ \n\ Create a bytearray object from a string of hexadecimal numbers.\n\ Spaces between two numbers are accepted.\n\ @@ -3121,10 +3125,10 @@ }; PyDoc_STRVAR(bytes_doc, -"bytearray(iterable_of_ints) -> bytearray.\n\ -bytearray(string, encoding[, errors]) -> bytearray.\n\ -bytearray(bytes_or_bytearray) -> mutable copy of bytes_or_bytearray.\n\ -bytearray(memory_view) -> bytearray.\n\ +"bytearray(iterable_of_ints) -> bytearray\n\ +bytearray(string, encoding[, errors]) -> bytearray\n\ +bytearray(bytes_or_bytearray) -> mutable copy of bytes_or_bytearray\n\ +bytearray(memory_view) -> bytearray\n\ \n\ Construct an mutable bytearray object from:\n\ - an iterable yielding integers in range(256)\n\ @@ -3132,7 +3136,7 @@ - a bytes or a bytearray object\n\ - any object implementing the buffer API.\n\ \n\ -bytearray(int) -> bytearray.\n\ +bytearray(int) -> bytearray\n\ \n\ Construct a zero-initialized bytearray of the given length."); Modified: python/branches/py3k/Objects/bytesobject.c ============================================================================== --- python/branches/py3k/Objects/bytesobject.c (original) +++ python/branches/py3k/Objects/bytesobject.c Fri May 30 10:20:09 2008 @@ -1208,7 +1208,7 @@ PyDoc_STRVAR(partition__doc__, "B.partition(sep) -> (head, sep, tail)\n\ \n\ -Searches for the separator sep in B, and returns the part before it,\n\ +Search for the separator sep in B, and return the part before it,\n\ the separator itself, and the part after it. If the separator is not\n\ found, returns B and two empty bytes objects."); @@ -1235,8 +1235,8 @@ PyDoc_STRVAR(rpartition__doc__, "B.rpartition(sep) -> (tail, sep, head)\n\ \n\ -Searches for the separator sep in B, starting at the end of B,\n\ -and returns the part before it, the separator itself, and the\n\ +Search for the separator sep in B, starting at the end of B,\n\ +and return the part before it, the separator itself, and the\n\ part after it. If the separator is not found, returns two empty\n\ bytes objects and B."); @@ -1423,7 +1423,7 @@ PyDoc_STRVAR(join__doc__, "B.join(iterable_of_bytes) -> bytes\n\ \n\ -Concatenates any number of bytes objects, with B in between each pair.\n\ +Concatenate any number of bytes objects, with B in between each pair.\n\ Example: b'.'.join([b'ab', b'pq', b'rs']) -> b'ab.pq.rs'."); static PyObject * @@ -1583,7 +1583,7 @@ PyDoc_STRVAR(find__doc__, -"B.find(sub [,start [,end]]) -> int\n\ +"B.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ such that sub is contained within s[start:end]. Optional\n\ @@ -1602,7 +1602,7 @@ PyDoc_STRVAR(index__doc__, -"B.index(sub [,start [,end]]) -> int\n\ +"B.index(sub[, start[, end]]) -> int\n\ \n\ Like B.find() but raise ValueError when the substring is not found."); @@ -1622,7 +1622,7 @@ PyDoc_STRVAR(rfind__doc__, -"B.rfind(sub [,start [,end]]) -> int\n\ +"B.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in B where substring sub is found,\n\ such that sub is contained within s[start:end]. Optional\n\ @@ -1641,7 +1641,7 @@ PyDoc_STRVAR(rindex__doc__, -"B.rindex(sub [,start [,end]]) -> int\n\ +"B.rindex(sub[, start[, end]]) -> int\n\ \n\ Like B.rfind() but raise ValueError when the substring is not found."); @@ -1792,7 +1792,7 @@ PyDoc_STRVAR(count__doc__, -"B.count(sub [,start [,end]]) -> int\n\ +"B.count(sub[, start[, end]]) -> int\n\ \n\ Return the number of non-overlapping occurrences of substring sub in\n\ string S[start:end]. Optional arguments start and end are interpreted\n\ @@ -2494,7 +2494,7 @@ } if (to_len == 0) { - /* delete all occurances of 'from' string */ + /* delete all occurrences of 'from' string */ if (from_len == 1) { return replace_delete_single_character( self, from_s[0], maxcount); @@ -2612,7 +2612,7 @@ PyDoc_STRVAR(startswith__doc__, -"B.startswith(prefix [,start [,end]]) -> bool\n\ +"B.startswith(prefix[, start[, end]]) -> bool\n\ \n\ Return True if B starts with the specified prefix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ @@ -2653,7 +2653,7 @@ PyDoc_STRVAR(endswith__doc__, -"B.endswith(suffix [,start [,end]]) -> bool\n\ +"B.endswith(suffix[, start[, end]]) -> bool\n\ \n\ Return True if B ends with the specified suffix, False otherwise.\n\ With optional start, test B beginning at that position.\n\ @@ -2694,9 +2694,9 @@ PyDoc_STRVAR(decode__doc__, -"B.decode([encoding[, errors]]) -> object\n\ +"B.decode([encoding[, errors]]) -> str\n\ \n\ -Decodes S using the codec registered for encoding. encoding defaults\n\ +Decode S using the codec registered for encoding. encoding defaults\n\ to the default encoding. errors may be given to set a different error\n\ handling scheme. Default is 'strict' meaning that encoding errors raise\n\ a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\ @@ -3027,10 +3027,10 @@ } PyDoc_STRVAR(string_doc, -"bytes(iterable_of_ints) -> bytes.\n\ +"bytes(iterable_of_ints) -> bytes\n\ bytes(string, encoding[, errors]) -> bytes\n\ -bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer.\n\ -bytes(memory_view) -> bytes.\n\ +bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer\n\ +bytes(memory_view) -> bytes\n\ \n\ Construct an immutable array of bytes from:\n\ - an iterable yielding integers in range(256)\n\ Modified: python/branches/py3k/Objects/unicodeobject.c ============================================================================== --- python/branches/py3k/Objects/unicodeobject.c (original) +++ python/branches/py3k/Objects/unicodeobject.c Fri May 30 10:20:09 2008 @@ -6167,7 +6167,7 @@ /* --- Unicode Object Methods --------------------------------------------- */ PyDoc_STRVAR(title__doc__, -"S.title() -> unicode\n\ +"S.title() -> str\n\ \n\ Return a titlecased version of S, i.e. words start with title case\n\ characters, all remaining cased characters have lower case."); @@ -6179,7 +6179,7 @@ } PyDoc_STRVAR(capitalize__doc__, -"S.capitalize() -> unicode\n\ +"S.capitalize() -> str\n\ \n\ Return a capitalized version of S, i.e. make the first character\n\ have upper case."); @@ -6192,7 +6192,7 @@ #if 0 PyDoc_STRVAR(capwords__doc__, -"S.capwords() -> unicode\n\ +"S.capwords() -> str\n\ \n\ Apply .capitalize() to all words in S and return the result with\n\ normalized whitespace (all whitespace strings are replaced by ' ')."); @@ -6256,7 +6256,7 @@ } PyDoc_STRVAR(center__doc__, -"S.center(width[, fillchar]) -> unicode\n\ +"S.center(width[, fillchar]) -> str\n\ \n\ Return S centered in a Unicode string of length width. Padding is\n\ done using the specified fill character (default is a space)"); @@ -6601,9 +6601,9 @@ } PyDoc_STRVAR(encode__doc__, -"S.encode([encoding[,errors]]) -> string or unicode\n\ +"S.encode([encoding[, errors]]) -> bytes\n\ \n\ -Encodes S using the codec registered for encoding. encoding defaults\n\ +Encode S using the codec registered for encoding. encoding defaults\n\ to the default encoding. errors may be given to set a different error\n\ handling scheme. Default is 'strict' meaning that encoding errors raise\n\ a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\ @@ -6637,7 +6637,7 @@ } PyDoc_STRVAR(expandtabs__doc__, -"S.expandtabs([tabsize]) -> unicode\n\ +"S.expandtabs([tabsize]) -> str\n\ \n\ Return a copy of S where all tab characters are expanded using spaces.\n\ If tabsize is not given, a tab size of 8 characters is assumed."); @@ -6724,7 +6724,7 @@ } PyDoc_STRVAR(find__doc__, -"S.find(sub [,start [,end]]) -> int\n\ +"S.find(sub[, start[, end]]) -> int\n\ \n\ Return the lowest index in S where substring sub is found,\n\ such that sub is contained within s[start:end]. Optional\n\ @@ -6789,7 +6789,7 @@ } PyDoc_STRVAR(index__doc__, -"S.index(sub [,start [,end]]) -> int\n\ +"S.index(sub[, start[, end]]) -> int\n\ \n\ Like S.find() but raise ValueError when the substring is not found."); @@ -7152,7 +7152,7 @@ } PyDoc_STRVAR(join__doc__, -"S.join(sequence) -> unicode\n\ +"S.join(sequence) -> str\n\ \n\ Return a string which is the concatenation of the strings in the\n\ sequence. The separator between elements is S."); @@ -7170,7 +7170,7 @@ } PyDoc_STRVAR(ljust__doc__, -"S.ljust(width[, fillchar]) -> int\n\ +"S.ljust(width[, fillchar]) -> str\n\ \n\ Return S left justified in a Unicode string of length width. Padding is\n\ done using the specified fill character (default is a space)."); @@ -7193,7 +7193,7 @@ } PyDoc_STRVAR(lower__doc__, -"S.lower() -> unicode\n\ +"S.lower() -> str\n\ \n\ Return a copy of the string S converted to lowercase."); @@ -7302,7 +7302,7 @@ PyDoc_STRVAR(strip__doc__, -"S.strip([chars]) -> unicode\n\ +"S.strip([chars]) -> str\n\ \n\ Return a copy of the string S with leading and trailing\n\ whitespace removed.\n\ @@ -7320,7 +7320,7 @@ PyDoc_STRVAR(lstrip__doc__, -"S.lstrip([chars]) -> unicode\n\ +"S.lstrip([chars]) -> str\n\ \n\ Return a copy of the string S with leading whitespace removed.\n\ If chars is given and not None, remove characters in chars instead.\n\ @@ -7337,7 +7337,7 @@ PyDoc_STRVAR(rstrip__doc__, -"S.rstrip([chars]) -> unicode\n\ +"S.rstrip([chars]) -> str\n\ \n\ Return a copy of the string S with trailing whitespace removed.\n\ If chars is given and not None, remove characters in chars instead.\n\ @@ -7444,7 +7444,7 @@ } PyDoc_STRVAR(replace__doc__, -"S.replace (old, new[, maxsplit]) -> unicode\n\ +"S.replace (old, new[, maxsplit]) -> str\n\ \n\ Return a copy of S with all occurrences of substring\n\ old replaced by new. If the optional argument maxsplit is\n\ @@ -7616,7 +7616,7 @@ } PyDoc_STRVAR(rfind__doc__, -"S.rfind(sub [,start [,end]]) -> int\n\ +"S.rfind(sub[, start[, end]]) -> int\n\ \n\ Return the highest index in S where substring sub is found,\n\ such that sub is contained within s[start:end]. Optional\n\ @@ -7647,7 +7647,7 @@ } PyDoc_STRVAR(rindex__doc__, -"S.rindex(sub [,start [,end]]) -> int\n\ +"S.rindex(sub[, start[, end]]) -> int\n\ \n\ Like S.rfind() but raise ValueError when the substring is not found."); @@ -7678,7 +7678,7 @@ } PyDoc_STRVAR(rjust__doc__, -"S.rjust(width[, fillchar]) -> unicode\n\ +"S.rjust(width[, fillchar]) -> str\n\ \n\ Return S right justified in a Unicode string of length width. Padding is\n\ done using the specified fill character (default is a space)."); @@ -7725,7 +7725,7 @@ } PyDoc_STRVAR(split__doc__, -"S.split([sep [,maxsplit]]) -> list of strings\n\ +"S.split([sep[, maxsplit]]) -> list of strings\n\ \n\ Return a list of the words in S, using sep as the\n\ delimiter string. If maxsplit is given, at most maxsplit\n\ @@ -7808,7 +7808,7 @@ PyDoc_STRVAR(partition__doc__, "S.partition(sep) -> (head, sep, tail)\n\ \n\ -Searches for the separator sep in S, and returns the part before it,\n\ +Search for the separator sep in S, and return the part before it,\n\ the separator itself, and the part after it. If the separator is not\n\ found, returns S and two empty strings."); @@ -7821,7 +7821,7 @@ PyDoc_STRVAR(rpartition__doc__, "S.rpartition(sep) -> (tail, sep, head)\n\ \n\ -Searches for the separator sep in S, starting at the end of S, and returns\n\ +Search for the separator sep in S, starting at the end of S, and return\n\ the part before it, the separator itself, and the part after it. If the\n\ separator is not found, returns two empty strings and S."); @@ -7856,7 +7856,7 @@ } PyDoc_STRVAR(rsplit__doc__, -"S.rsplit([sep [,maxsplit]]) -> list of strings\n\ +"S.rsplit([sep[, maxsplit]]) -> list of strings\n\ \n\ Return a list of the words in S, using sep as the\n\ delimiter string, starting at the end of the string and\n\ @@ -7912,7 +7912,7 @@ } PyDoc_STRVAR(swapcase__doc__, -"S.swapcase() -> unicode\n\ +"S.swapcase() -> str\n\ \n\ Return a copy of S with uppercase characters converted to lowercase\n\ and vice versa."); @@ -8027,7 +8027,7 @@ } PyDoc_STRVAR(translate__doc__, -"S.translate(table) -> unicode\n\ +"S.translate(table) -> str\n\ \n\ Return a copy of the string S, where all characters have been mapped\n\ through the given translation table, which must be a mapping of\n\ @@ -8042,7 +8042,7 @@ } PyDoc_STRVAR(upper__doc__, -"S.upper() -> unicode\n\ +"S.upper() -> str\n\ \n\ Return a copy of S converted to uppercase."); @@ -8053,7 +8053,7 @@ } PyDoc_STRVAR(zfill__doc__, -"S.zfill(width) -> unicode\n\ +"S.zfill(width) -> str\n\ \n\ Pad a numeric string x with zeros on the left, to fill a field\n\ of the specified width. The string x is never truncated."); @@ -8198,12 +8198,12 @@ #include "stringlib/string_format.h" PyDoc_STRVAR(format__doc__, -"S.format(*args, **kwargs) -> unicode\n\ +"S.format(*args, **kwargs) -> str\n\ \n\ "); PyDoc_STRVAR(p_format__doc__, -"S.__format__(format_spec) -> unicode\n\ +"S.__format__(format_spec) -> str\n\ \n\ "); @@ -9108,7 +9108,7 @@ } PyDoc_STRVAR(unicode_doc, -"str(string [, encoding[, errors]]) -> object\n\ +"str(string[, encoding[, errors]]) -> str\n\ \n\ Create a new string object from the given encoded string.\n\ encoding defaults to the current default string encoding.\n\ From python-3000-checkins at python.org Fri May 30 14:02:38 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Fri, 30 May 2008 14:02:38 +0200 (CEST) Subject: [Python-3000-checkins] r63810 - python/branches/py3k/Objects/bytearrayobject.c Message-ID: <20080530120238.478D11E4005@bag.python.org> Author: georg.brandl Date: Fri May 30 14:02:38 2008 New Revision: 63810 Log: Don't return the same object from trivial replace() cases -- it is guaranteed to return a new bytearray. Modified: python/branches/py3k/Objects/bytearrayobject.c Modified: python/branches/py3k/Objects/bytearrayobject.c ============================================================================== --- python/branches/py3k/Objects/bytearrayobject.c (original) +++ python/branches/py3k/Objects/bytearrayobject.c Fri May 30 14:02:38 2008 @@ -1459,10 +1459,7 @@ Py_LOCAL(PyByteArrayObject *) return_self(PyByteArrayObject *self) { - if (PyByteArray_CheckExact(self)) { - Py_INCREF(self); - return (PyByteArrayObject *)self; - } + /* always return a new bytearray */ return (PyByteArrayObject *)PyByteArray_FromStringAndSize( PyByteArray_AS_STRING(self), PyByteArray_GET_SIZE(self)); From python-3000-checkins at python.org Fri May 30 14:05:02 2008 From: python-3000-checkins at python.org (georg.brandl) Date: Fri, 30 May 2008 14:05:02 +0200 (CEST) Subject: [Python-3000-checkins] r63811 - python/branches/py3k/Lib/test/test_bytes.py Message-ID: <20080530120502.86A321E400A@bag.python.org> Author: georg.brandl Date: Fri May 30 14:05:02 2008 New Revision: 63811 Log: Add test case for r63810. Modified: python/branches/py3k/Lib/test/test_bytes.py Modified: python/branches/py3k/Lib/test/test_bytes.py ============================================================================== --- python/branches/py3k/Lib/test/test_bytes.py (original) +++ python/branches/py3k/Lib/test/test_bytes.py Fri May 30 14:05:02 2008 @@ -832,6 +832,11 @@ def test_rsplit_bytearray(self): self.assertEqual(b'a b'.rsplit(memoryview(b' ')), [b'a', b'b']) + def test_return_self(self): + # bytearray.replace must always return a new bytearray + b = bytearray() + self.failIf(b.replace(b'', b'') is b) + # Optimizations: # __iter__? (optimization) # __reversed__? (optimization) From python-3000-checkins at python.org Fri May 30 20:10:19 2008 From: python-3000-checkins at python.org (eric.smith) Date: Fri, 30 May 2008 20:10:19 +0200 (CEST) Subject: [Python-3000-checkins] r63815 - in python/branches/py3k: Include/floatobject.h Include/formatter_unicode.h Include/longobject.h Include/unicodeobject.h Makefile.pre.in Objects/floatobject.c Objects/longobject.c Objects/stringlib/formatter.h Objects/unicodeobject.c Python/formatter_unicode.c Message-ID: <20080530181019.C43801E4005@bag.python.org> Author: eric.smith Date: Fri May 30 20:10:19 2008 New Revision: 63815 Log: Refactor and clean up str.format() code (and helpers) in advance of optimizations. Removed: python/branches/py3k/Include/formatter_unicode.h Modified: python/branches/py3k/Include/floatobject.h python/branches/py3k/Include/longobject.h python/branches/py3k/Include/unicodeobject.h python/branches/py3k/Makefile.pre.in python/branches/py3k/Objects/floatobject.c python/branches/py3k/Objects/longobject.c python/branches/py3k/Objects/stringlib/formatter.h python/branches/py3k/Objects/unicodeobject.c python/branches/py3k/Python/formatter_unicode.c Modified: python/branches/py3k/Include/floatobject.h ============================================================================== --- python/branches/py3k/Include/floatobject.h (original) +++ python/branches/py3k/Include/floatobject.h Fri May 30 20:10:19 2008 @@ -105,6 +105,12 @@ /* free list api */ PyAPI_FUNC(void) PyFloat_CompactFreeList(size_t *, size_t *, size_t *); +/* Format the object based on the format_spec, as defined in PEP 3101 + (Advanced String Formatting). */ +PyAPI_FUNC(PyObject *) _PyFloat_FormatAdvanced(PyObject *obj, + Py_UNICODE *format_spec, + Py_ssize_t format_spec_len); + #ifdef __cplusplus } #endif Deleted: python/branches/py3k/Include/formatter_unicode.h ============================================================================== --- python/branches/py3k/Include/formatter_unicode.h Fri May 30 20:10:19 2008 +++ (empty file) @@ -1,9 +0,0 @@ -PyObject * -unicode_unicode__format__(PyObject *self, PyObject *args); - -PyObject * -unicode_long__format__(PyObject *self, PyObject *args); - -PyObject * -unicode_float__format__(PyObject *self, PyObject *args); - Modified: python/branches/py3k/Include/longobject.h ============================================================================== --- python/branches/py3k/Include/longobject.h (original) +++ python/branches/py3k/Include/longobject.h Fri May 30 20:10:19 2008 @@ -127,6 +127,12 @@ appending a base prefix of 0[box] if base is 2, 8 or 16. */ PyAPI_FUNC(PyObject *) _PyLong_Format(PyObject *aa, int base); +/* Format the object based on the format_spec, as defined in PEP 3101 + (Advanced String Formatting). */ +PyAPI_FUNC(PyObject *) _PyLong_FormatAdvanced(PyObject *obj, + Py_UNICODE *format_spec, + Py_ssize_t format_spec_len); + /* These aren't really part of the long object, but they're handy. The functions are in Python/mystrtoul.c. */ Modified: python/branches/py3k/Include/unicodeobject.h ============================================================================== --- python/branches/py3k/Include/unicodeobject.h (original) +++ python/branches/py3k/Include/unicodeobject.h Fri May 30 20:10:19 2008 @@ -571,6 +571,12 @@ PyAPI_FUNC(PyObject *) PyUnicode_FromFormatV(const char*, va_list); PyAPI_FUNC(PyObject *) PyUnicode_FromFormat(const char*, ...); +/* Format the object based on the format_spec, as defined in PEP 3101 + (Advanced String Formatting). */ +PyAPI_FUNC(PyObject *) _PyUnicode_FormatAdvanced(PyObject *obj, + Py_UNICODE *format_spec, + Py_ssize_t format_spec_len); + PyAPI_FUNC(void) PyUnicode_InternInPlace(PyObject **); PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **); PyAPI_FUNC(PyObject *) PyUnicode_InternFromString(const char *); Modified: python/branches/py3k/Makefile.pre.in ============================================================================== --- python/branches/py3k/Makefile.pre.in (original) +++ python/branches/py3k/Makefile.pre.in Fri May 30 20:10:19 2008 @@ -596,7 +596,6 @@ Include/eval.h \ Include/fileobject.h \ Include/floatobject.h \ - Include/formatter_unicode.h \ Include/frameobject.h \ Include/funcobject.h \ Include/genobject.h \ Modified: python/branches/py3k/Objects/floatobject.c ============================================================================== --- python/branches/py3k/Objects/floatobject.c (original) +++ python/branches/py3k/Objects/floatobject.c Fri May 30 20:10:19 2008 @@ -7,8 +7,6 @@ #include "Python.h" #include "structseq.h" -#include "formatter_unicode.h" - #include #include @@ -1303,10 +1301,13 @@ static PyObject * float__format__(PyObject *self, PyObject *args) { - /* when back porting this to 2.6, check type of the format_spec - and call either unicode_long__format__ or - string_long__format__ */ - return unicode_float__format__(self, args); + PyObject *format_spec; + + if (!PyArg_ParseTuple(args, "U:__format__", &format_spec)) + return NULL; + return _PyFloat_FormatAdvanced(self, + PyUnicode_AS_UNICODE(format_spec), + PyUnicode_GET_SIZE(format_spec)); } PyDoc_STRVAR(float__format__doc, Modified: python/branches/py3k/Objects/longobject.c ============================================================================== --- python/branches/py3k/Objects/longobject.c (original) +++ python/branches/py3k/Objects/longobject.c Fri May 30 20:10:19 2008 @@ -5,8 +5,6 @@ #include "Python.h" #include "longintrepr.h" -#include "formatter_unicode.h" - #include #ifndef NSMALLPOSINTS @@ -3590,10 +3588,13 @@ static PyObject * long__format__(PyObject *self, PyObject *args) { - /* when back porting this to 2.6, check type of the format_spec - and call either unicode_long__format__ or - string_long__format__ */ - return unicode_long__format__(self, args); + PyObject *format_spec; + + if (!PyArg_ParseTuple(args, "U:__format__", &format_spec)) + return NULL; + return _PyLong_FormatAdvanced(self, + PyUnicode_AS_UNICODE(format_spec), + PyUnicode_GET_SIZE(format_spec)); } Modified: python/branches/py3k/Objects/stringlib/formatter.h ============================================================================== --- python/branches/py3k/Objects/stringlib/formatter.h (original) +++ python/branches/py3k/Objects/stringlib/formatter.h Fri May 30 20:10:19 2008 @@ -102,12 +102,13 @@ if failure, sets the exception */ static int -parse_internal_render_format_spec(PyObject *format_spec, +parse_internal_render_format_spec(STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len, InternalFormatSpec *format, char default_type) { - STRINGLIB_CHAR *ptr = STRINGLIB_STR(format_spec); - STRINGLIB_CHAR *end = ptr + STRINGLIB_LEN(format_spec); + STRINGLIB_CHAR *ptr = format_spec; + STRINGLIB_CHAR *end = format_spec + format_spec_len; /* end-ptr is used throughout this code to specify the length of the input string */ @@ -756,56 +757,31 @@ /************************************************************************/ /*********** built in formatters ****************************************/ /************************************************************************/ -#ifdef FORMAT_STRING PyObject * -FORMAT_STRING(PyObject* value, PyObject* args) +FORMAT_STRING(PyObject *obj, + STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len) { - PyObject *format_spec; - PyObject *result = NULL; -#if PY_VERSION_HEX < 0x03000000 - PyObject *tmp = NULL; -#endif InternalFormatSpec format; - - /* If 2.x, we accept either str or unicode, and try to convert it - to the right type. In 3.x, we insist on only unicode */ -#if PY_VERSION_HEX >= 0x03000000 - if (!PyArg_ParseTuple(args, STRINGLIB_PARSE_CODE ":__format__", - &format_spec)) - goto done; -#else - /* If 2.x, convert format_spec to the same type as value */ - /* This is to allow things like u''.format('') */ - if (!PyArg_ParseTuple(args, "O:__format__", &format_spec)) - goto done; - if (!(PyBytes_Check(format_spec) || PyUnicode_Check(format_spec))) { - PyErr_Format(PyExc_TypeError, "__format__ arg must be str " - "or unicode, not %s", Py_TYPE(format_spec)->tp_name); - goto done; - } - tmp = STRINGLIB_TOSTR(format_spec); - if (tmp == NULL) - goto done; - format_spec = tmp; -#endif + PyObject *result = NULL; /* check for the special case of zero length format spec, make - it equivalent to str(value) */ - if (STRINGLIB_LEN(format_spec) == 0) { - result = STRINGLIB_TOSTR(value); + it equivalent to str(obj) */ + if (format_spec_len == 0) { + result = STRINGLIB_TOSTR(obj); goto done; } - /* parse the format_spec */ - if (!parse_internal_render_format_spec(format_spec, &format, 's')) + if (!parse_internal_render_format_spec(format_spec, format_spec_len, + &format, 's')) goto done; /* type conversion? */ switch (format.type) { case 's': /* no type conversion needed, already a string. do the formatting */ - result = format_string_internal(value, &format); + result = format_string_internal(obj, &format); break; default: /* unknown */ @@ -826,35 +802,31 @@ } done: -#if PY_VERSION_HEX < 0x03000000 - Py_XDECREF(tmp); -#endif return result; } -#endif /* FORMAT_STRING */ #if defined FORMAT_LONG || defined FORMAT_INT static PyObject* -format_int_or_long(PyObject* value, PyObject* args, IntOrLongToString tostring) +format_int_or_long(PyObject* obj, + STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len, + IntOrLongToString tostring) { - PyObject *format_spec; PyObject *result = NULL; PyObject *tmp = NULL; InternalFormatSpec format; - if (!PyArg_ParseTuple(args, STRINGLIB_PARSE_CODE ":__format__", - &format_spec)) - goto done; - /* check for the special case of zero length format spec, make - it equivalent to str(value) */ - if (STRINGLIB_LEN(format_spec) == 0) { - result = STRINGLIB_TOSTR(value); + it equivalent to str(obj) */ + if (format_spec_len == 0) { + result = STRINGLIB_TOSTR(obj); goto done; } /* parse the format_spec */ - if (!parse_internal_render_format_spec(format_spec, &format, 'd')) + if (!parse_internal_render_format_spec(format_spec, + format_spec_len, + &format, 'd')) goto done; /* type conversion? */ @@ -868,7 +840,7 @@ case 'n': /* no type conversion needed, already an int (or long). do the formatting */ - result = format_int_or_long_internal(value, &format, tostring); + result = format_int_or_long_internal(obj, &format, tostring); break; case 'e': @@ -879,10 +851,10 @@ case 'G': case '%': /* convert to float */ - tmp = PyNumber_Float(value); + tmp = PyNumber_Float(obj); if (tmp == NULL) goto done; - result = format_float_internal(value, &format); + result = format_float_internal(obj, &format); break; default: @@ -917,9 +889,12 @@ #endif PyObject * -FORMAT_LONG(PyObject* value, PyObject* args) +FORMAT_LONG(PyObject *obj, + STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len) { - return format_int_or_long(value, args, long_format); + return format_int_or_long(obj, format_spec, format_spec_len, + long_format); } #endif /* FORMAT_LONG */ @@ -935,32 +910,35 @@ } PyObject * -FORMAT_INT(PyObject* value, PyObject* args) +FORMAT_INT(PyObject *obj, + STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len) { - return format_int_or_long(value, args, int_format); + return format_int_or_long(obj, format_spec, format_spec_len, + int_format); } #endif /* FORMAT_INT */ #ifdef FORMAT_FLOAT PyObject * -FORMAT_FLOAT(PyObject *value, PyObject *args) +FORMAT_FLOAT(PyObject *obj, + STRINGLIB_CHAR *format_spec, + Py_ssize_t format_spec_len) { - PyObject *format_spec; PyObject *result = NULL; InternalFormatSpec format; - if (!PyArg_ParseTuple(args, STRINGLIB_PARSE_CODE ":__format__", &format_spec)) - goto done; - /* check for the special case of zero length format spec, make - it equivalent to str(value) */ - if (STRINGLIB_LEN(format_spec) == 0) { - result = STRINGLIB_TOSTR(value); + it equivalent to str(obj) */ + if (format_spec_len == 0) { + result = STRINGLIB_TOSTR(obj); goto done; } /* parse the format_spec */ - if (!parse_internal_render_format_spec(format_spec, &format, '\0')) + if (!parse_internal_render_format_spec(format_spec, + format_spec_len, + &format, '\0')) goto done; /* type conversion? */ @@ -979,7 +957,7 @@ case 'n': case '%': /* no conversion, already a float. do the formatting */ - result = format_float_internal(value, &format); + result = format_float_internal(obj, &format); break; default: Modified: python/branches/py3k/Objects/unicodeobject.c ============================================================================== --- python/branches/py3k/Objects/unicodeobject.c (original) +++ python/branches/py3k/Objects/unicodeobject.c Fri May 30 20:10:19 2008 @@ -46,8 +46,6 @@ #include "unicodeobject.h" #include "ucnhash.h" -#include "formatter_unicode.h" - #ifdef MS_WINDOWS #include #endif @@ -8202,6 +8200,19 @@ \n\ "); +static PyObject * +unicode__format__(PyObject* self, PyObject* args) +{ + PyObject *format_spec; + + if (!PyArg_ParseTuple(args, "U:__format__", &format_spec)) + return NULL; + + return _PyUnicode_FormatAdvanced(self, + PyUnicode_AS_UNICODE(format_spec), + PyUnicode_GET_SIZE(format_spec)); +} + PyDoc_STRVAR(p_format__doc__, "S.__format__(format_spec) -> str\n\ \n\ @@ -8259,7 +8270,7 @@ {"isidentifier", (PyCFunction) unicode_isidentifier, METH_NOARGS, isidentifier__doc__}, {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__}, {"format", (PyCFunction) do_string_format, METH_VARARGS | METH_KEYWORDS, format__doc__}, - {"__format__", (PyCFunction) unicode_unicode__format__, METH_VARARGS, p_format__doc__}, + {"__format__", (PyCFunction) unicode__format__, METH_VARARGS, p_format__doc__}, {"_formatter_field_name_split", (PyCFunction) formatter_field_name_split, METH_NOARGS}, {"_formatter_parser", (PyCFunction) formatter_parser, METH_NOARGS}, {"maketrans", (PyCFunction) unicode_maketrans, Modified: python/branches/py3k/Python/formatter_unicode.c ============================================================================== --- python/branches/py3k/Python/formatter_unicode.c (original) +++ python/branches/py3k/Python/formatter_unicode.c Fri May 30 20:10:19 2008 @@ -3,11 +3,11 @@ of int.__float__, etc., that take and return unicode objects */ #include "Python.h" -#include "formatter_unicode.h" - #include "../Objects/stringlib/unicodedefs.h" -#define FORMAT_STRING unicode_unicode__format__ -#define FORMAT_LONG unicode_long__format__ -#define FORMAT_FLOAT unicode_float__format__ + +#define FORMAT_STRING _PyUnicode_FormatAdvanced +#define FORMAT_LONG _PyLong_FormatAdvanced +#define FORMAT_FLOAT _PyFloat_FormatAdvanced + #include "../Objects/stringlib/formatter.h" From python-3000-checkins at python.org Fri May 30 20:12:26 2008 From: python-3000-checkins at python.org (eric.smith) Date: Fri, 30 May 2008 20:12:26 +0200 (CEST) Subject: [Python-3000-checkins] r63816 - python/branches/py3k Message-ID: <20080530181226.BA5531E4006@bag.python.org> Author: eric.smith Date: Fri May 30 20:12:26 2008 New Revision: 63816 Log: Blocked revisions 63814 via svnmerge ........ r63814 | eric.smith | 2008-05-30 14:10:04 -0400 (Fri, 30 May 2008) | 1 line Refactor and clean up str.format() code (and helpers) in advance of optimizations. ........ Modified: python/branches/py3k/ (props changed) From python-3000-checkins at python.org Sat May 31 02:21:00 2008 From: python-3000-checkins at python.org (benjamin.peterson) Date: Sat, 31 May 2008 02:21:00 +0200 (CEST) Subject: [Python-3000-checkins] r63825 - in python/branches/py3k: Makefile.pre.in configure configure.in Message-ID: <20080531002100.1FCBF1E4006@bag.python.org> Author: benjamin.peterson Date: Sat May 31 02:20:59 2008 New Revision: 63825 Log: remove plat-mac references Modified: python/branches/py3k/Makefile.pre.in python/branches/py3k/configure python/branches/py3k/configure.in Modified: python/branches/py3k/Makefile.pre.in ============================================================================== --- python/branches/py3k/Makefile.pre.in (original) +++ python/branches/py3k/Makefile.pre.in Sat May 31 02:20:59 2008 @@ -803,16 +803,6 @@ EXTRAMACHDEPPATH=@EXTRAMACHDEPPATH@ MACHDEPS= $(PLATDIR) $(EXTRAPLATDIR) XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax -PLATMACDIRS= plat-mac plat-mac/Carbon plat-mac/lib-scriptpackages \ - plat-mac/lib-scriptpackages/_builtinSuites \ - plat-mac/lib-scriptpackages/CodeWarrior \ - plat-mac/lib-scriptpackages/Explorer \ - plat-mac/lib-scriptpackages/Finder \ - plat-mac/lib-scriptpackages/Netscape \ - plat-mac/lib-scriptpackages/StdSuites \ - plat-mac/lib-scriptpackages/SystemEvents \ - plat-mac/lib-scriptpackages/Terminal -PLATMACPATH=:plat-mac:plat-mac/lib-scriptpackages LIBSUBDIRS= tkinter site-packages test test/output test/data \ test/decimaltestdata \ encodings \ Modified: python/branches/py3k/configure ============================================================================== --- python/branches/py3k/configure (original) +++ python/branches/py3k/configure Sat May 31 02:20:59 2008 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 63205 . +# From configure.in Revision: 63697 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.61 for python 3.0. # @@ -669,8 +669,6 @@ FRAMEWORKUNIXTOOLSPREFIX MACHDEP SGI_ABI -EXTRAPLATDIR -EXTRAMACHDEPPATH CONFIGURE_MACOSX_DEPLOYMENT_TARGET EXPORT_MACOSX_DEPLOYMENT_TARGET CC @@ -2146,27 +2144,6 @@ { echo "$as_me:$LINENO: result: $MACHDEP" >&5 echo "${ECHO_T}$MACHDEP" >&6; } -# And add extra plat-mac for darwin - - -{ echo "$as_me:$LINENO: checking EXTRAPLATDIR" >&5 -echo $ECHO_N "checking EXTRAPLATDIR... $ECHO_C" >&6; } -if test -z "$EXTRAPLATDIR" -then - case $MACHDEP in - darwin) - EXTRAPLATDIR="\$(PLATMACDIRS)" - EXTRAMACHDEPPATH="\$(PLATMACPATH)" - ;; - *) - EXTRAPLATDIR="" - EXTRAMACHDEPPATH="" - ;; - esac -fi -{ echo "$as_me:$LINENO: result: $EXTRAPLATDIR" >&5 -echo "${ECHO_T}$EXTRAPLATDIR" >&6; } - # Record the configure-time value of MACOSX_DEPLOYMENT_TARGET, # it may influence the way we can build extensions, so distutils # needs to check it @@ -24878,8 +24855,6 @@ FRAMEWORKUNIXTOOLSPREFIX!$FRAMEWORKUNIXTOOLSPREFIX$ac_delim MACHDEP!$MACHDEP$ac_delim SGI_ABI!$SGI_ABI$ac_delim -EXTRAPLATDIR!$EXTRAPLATDIR$ac_delim -EXTRAMACHDEPPATH!$EXTRAMACHDEPPATH$ac_delim CONFIGURE_MACOSX_DEPLOYMENT_TARGET!$CONFIGURE_MACOSX_DEPLOYMENT_TARGET$ac_delim EXPORT_MACOSX_DEPLOYMENT_TARGET!$EXPORT_MACOSX_DEPLOYMENT_TARGET$ac_delim CC!$CC$ac_delim @@ -24922,6 +24897,8 @@ CFLAGSFORSHARED!$CFLAGSFORSHARED$ac_delim SHLIBS!$SHLIBS$ac_delim USE_SIGNAL_MODULE!$USE_SIGNAL_MODULE$ac_delim +SIGNAL_OBJS!$SIGNAL_OBJS$ac_delim +USE_THREAD_MODULE!$USE_THREAD_MODULE$ac_delim _ACEOF if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 97; then @@ -24963,8 +24940,6 @@ ac_delim='%!_!# ' for ac_last_try in false false false false false :; do cat >conf$$subs.sed <<_ACEOF -SIGNAL_OBJS!$SIGNAL_OBJS$ac_delim -USE_THREAD_MODULE!$USE_THREAD_MODULE$ac_delim LDLAST!$LDLAST$ac_delim THREADOBJ!$THREADOBJ$ac_delim DLINCLDIR!$DLINCLDIR$ac_delim @@ -24984,7 +24959,7 @@ LTLIBOBJS!$LTLIBOBJS$ac_delim _ACEOF - if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 19; then + if test `sed -n "s/.*$ac_delim\$/X/p" conf$$subs.sed | grep -c X` = 17; then break elif $ac_last_try; then { { echo "$as_me:$LINENO: error: could not make $CONFIG_STATUS" >&5 Modified: python/branches/py3k/configure.in ============================================================================== --- python/branches/py3k/configure.in (original) +++ python/branches/py3k/configure.in Sat May 31 02:20:59 2008 @@ -322,25 +322,6 @@ fi AC_MSG_RESULT($MACHDEP) -# And add extra plat-mac for darwin -AC_SUBST(EXTRAPLATDIR) -AC_SUBST(EXTRAMACHDEPPATH) -AC_MSG_CHECKING(EXTRAPLATDIR) -if test -z "$EXTRAPLATDIR" -then - case $MACHDEP in - darwin) - EXTRAPLATDIR="\$(PLATMACDIRS)" - EXTRAMACHDEPPATH="\$(PLATMACPATH)" - ;; - *) - EXTRAPLATDIR="" - EXTRAMACHDEPPATH="" - ;; - esac -fi -AC_MSG_RESULT($EXTRAPLATDIR) - # Record the configure-time value of MACOSX_DEPLOYMENT_TARGET, # it may influence the way we can build extensions, so distutils # needs to check it From python-3000-checkins at python.org Sat May 31 03:40:09 2008 From: python-3000-checkins at python.org (eric.smith) Date: Sat, 31 May 2008 03:40:09 +0200 (CEST) Subject: [Python-3000-checkins] r63826 - python/branches/py3k/Objects/stringlib/string_format.h Message-ID: <20080531014009.124D01E400B@bag.python.org> Author: eric.smith Date: Sat May 31 03:40:08 2008 New Revision: 63826 Log: Optimization of str.format() for cases with unicode, long, and float arguments. This gives about 30% speed improvement for the simplest (but most common) cases. This patch skips the __format__ dispatch, and also avoids creating an object to hold the format_spec. Unfortunately, backporting this to 2.6 is going to be more challenging due to str/unicode issues with format_spec. I'll work on that next. Then I'll spend some time profiling and see what that tells me. Modified: python/branches/py3k/Objects/stringlib/string_format.h Modified: python/branches/py3k/Objects/stringlib/string_format.h ============================================================================== --- python/branches/py3k/Objects/stringlib/string_format.h (original) +++ python/branches/py3k/Objects/stringlib/string_format.h Sat May 31 03:40:08 2008 @@ -483,13 +483,34 @@ { int ok = 0; PyObject *result = NULL; + PyObject *format_spec_object = NULL; - /* we need to create an object out of the pointers we have */ - PyObject *format_spec_object = SubString_new_object_or_empty(format_spec); - if (format_spec_object == NULL) - goto done; + STRINGLIB_CHAR* format_spec_start = format_spec->ptr ? + format_spec->ptr : NULL; + Py_ssize_t format_spec_len = format_spec->ptr ? + format_spec->end - format_spec->ptr : 0; + + /* If we know the type exactly, skip the lookup of __format__ and just + call the formatter directly. */ + if (PyUnicode_CheckExact(fieldobj)) + result = _PyUnicode_FormatAdvanced(fieldobj, format_spec_start, + format_spec_len); + else if (PyLong_CheckExact(fieldobj)) + result = _PyLong_FormatAdvanced(fieldobj, format_spec_start, + format_spec_len); + else if (PyFloat_CheckExact(fieldobj)) + result = _PyFloat_FormatAdvanced(fieldobj, format_spec_start, + format_spec_len); + else { + /* We need to create an object out of the pointers we have, because + __format__ takes a string/unicode object for format_spec. */ + format_spec_object = STRINGLIB_NEW(format_spec_start, + format_spec_len); + if (format_spec_object == NULL) + goto done; - result = PyObject_Format(fieldobj, format_spec_object); + result = PyObject_Format(fieldobj, format_spec_object); + } if (result == NULL) goto done; @@ -512,7 +533,7 @@ ok = output_data(output, STRINGLIB_STR(result), STRINGLIB_LEN(result)); done: - Py_DECREF(format_spec_object); + Py_XDECREF(format_spec_object); Py_XDECREF(result); return ok; } From python-3000-checkins at python.org Sat May 31 15:05:35 2008 From: python-3000-checkins at python.org (mark.summerfield) Date: Sat, 31 May 2008 15:05:35 +0200 (CEST) Subject: [Python-3000-checkins] r63829 - in python: branches/py3k/Doc/library/re.rst trunk/Doc/library/re.rst Message-ID: <20080531130535.5DCD11E4004@bag.python.org> Author: mark.summerfield Date: Sat May 31 15:05:34 2008 New Revision: 63829 Log: Added a note to [] that special forms & special chars lose their meaning and backrefs can't be used inside [] Modified: python/branches/py3k/Doc/library/re.rst Changes in other areas also in this revision: Modified: python/trunk/Doc/library/re.rst Modified: python/branches/py3k/Doc/library/re.rst ============================================================================== --- python/branches/py3k/Doc/library/re.rst (original) +++ python/branches/py3k/Doc/library/re.rst Sat May 31 15:05:34 2008 @@ -181,6 +181,12 @@ ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any character except ``'^'``. + Note that inside ``[]`` the special forms and special characters lose + their meanings and only the syntaxes described here are valid. For + example, ``+``, ``*``, ``(``, ``)``, and so on are treated as + literals inside ``[]``, and backreferences cannot be used inside + ``[]``. + ``'|'`` ``A|B``, where A and B can be arbitrary REs, creates a regular expression that will match either A or B. An arbitrary number of REs can be separated by the Diffs of changes in other areas also in this revision: Modified: python/trunk/Doc/library/re.rst ============================================================================== --- python/trunk/Doc/library/re.rst (original) +++ python/trunk/Doc/library/re.rst Sat May 31 15:05:34 2008 @@ -181,6 +181,12 @@ ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any character except ``'^'``. + Note that inside ``[]`` the special forms and special characters lose + their meanings and only the syntaxes described here are valid. For + example, ``+``, ``*``, ``(``, ``)``, and so on are treated as + literals inside ``[]``, and backreferences cannot be used inside + ``[]``. + ``'|'`` ``A|B``, where A and B can be arbitrary REs, creates a regular expression that will match either A or B. An arbitrary number of REs can be separated by the