[Python-checkins] cpython (merge default -> default): Merged upstream changes.
vinay.sajip
python-checkins at python.org
Sun Jun 24 12:24:13 CEST 2012
http://hg.python.org/cpython/rev/a9e1bf656fbc
changeset: 77691:a9e1bf656fbc
parent: 77690:4a3439ef552c
parent: 77689:8993160609f5
user: Vinay Sajip <vinay_sajip at yahoo.co.uk>
date: Sun Jun 24 11:24:05 2012 +0100
summary:
Merged upstream changes.
files:
Doc/library/email.headerregistry.rst | 2 +-
Doc/library/os.rst | 1052 ++++++-------
Lib/test/test_random.py | 6 +-
Misc/NEWS | 4 +
Modules/_decimal/_decimal.c | 89 +-
Modules/_randommodule.c | 9 +-
6 files changed, 541 insertions(+), 621 deletions(-)
diff --git a/Doc/library/email.headerregistry.rst b/Doc/library/email.headerregistry.rst
--- a/Doc/library/email.headerregistry.rst
+++ b/Doc/library/email.headerregistry.rst
@@ -268,7 +268,7 @@
also take a list of supplemental parameters, which have a common format.
This class serves as a base for all the MIME headers that take parameters.
- .. attrbibute:: params
+ .. attribute:: params
A dictionary mapping parameter names to parameter values.
diff --git a/Doc/library/os.rst b/Doc/library/os.rst
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -645,7 +645,7 @@
.. function:: closerange(fd_low, fd_high)
Close all file descriptors from *fd_low* (inclusive) to *fd_high* (exclusive),
- ignoring errors. Equivalent to::
+ ignoring errors. Equivalent to (but much faster than)::
for fd in range(fd_low, fd_high):
try:
@@ -729,6 +729,7 @@
Availability: Unix, Windows.
+
.. function:: fstatvfs(fd)
Return information about the filesystem containing the file associated with file
@@ -823,13 +824,8 @@
this module too (see :ref:`open-constants`). In particular, on Windows adding
:const:`O_BINARY` is needed to open files in binary mode.
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
+ This function can support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
Availability: Unix, Windows.
@@ -1171,6 +1167,49 @@
Files and Directories
---------------------
+On some Unix platforms, many of these functions support one or more of these
+features:
+
+.. _path_fd:
+
+* For some functions, the *path* argument can be not only a string giving a path
+ name, but also a file descriptor. The function will then operate on the file
+ referred to by the descriptor. (For POSIX systems, this will use the ``f...``
+ versions of the function.)
+
+ You can check whether or not *path* can be specified as a file descriptor on
+ your platform using :data:`os.supports_fd`. If it is unavailable, using it
+ will raise a :exc:`NotImplementedError`.
+
+ If the function also supports *dir_fd* or *follow_symlinks* arguments, it is
+ an error to specify one of those when supplying *path* as a file descriptor.
+
+.. _dir_fd:
+
+* For functions with a *dir_fd* parameter: If *dir_fd* is not ``None``, it
+ should be a file descriptor referring to a directory, and the path to operate
+ on should be relative; path will then be relative to that directory. If the
+ path is absolute, *dir_fd* is ignored. (For POSIX systems, this will use the
+ ``f...at`` versions of the function.)
+
+ You can check whether or not *dir_fd* is supported on your platform using
+ :data:`os.supports_dir_fd`. If it is unavailable, using it will raise a
+ :exc:`NotImplementedError`.
+
+.. _follow_symlinks:
+
+* For functions ith a *follow_symlinks* parameter: If *follow_symlinks* is
+ ``False``, and the last element of the path to operate on is a symbolic link,
+ the function will operate on the symbolic link itself instead of the file the
+ link points to. (For POSIX systems, this will use the ``l...`` versions of
+ the function.)
+
+ You can check whether or not *follow_symlinks* is supported on your platform
+ using :data:`os.supports_follow_symlinks`. If it is unavailable, using it
+ will raise a :exc:`NotImplementedError`.
+
+
+
.. function:: access(path, mode, *, dir_fd=None, effective_ids=False, follow_symlinks=True)
Use the real uid/gid to test for access to *path*. Note that most operations
@@ -1182,13 +1221,8 @@
:const:`False` if not. See the Unix man page :manpage:`access(2)` for more
information.
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
+ This function can support specifying :ref:`paths relative to directory
+ descriptors <dir_fd>` and :ref:`not following symlinks <follow_symlinks>`.
If *effective_ids* is ``True``, :func:`access` will perform its access
checks using the effective uid/gid instead of the real uid/gid.
@@ -1196,13 +1230,6 @@
or not it is available using :data:`os.supports_effective_ids`. If it is
unavailable, using it will raise a :exc:`NotImplementedError`.
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`access` will examine the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
-
Availability: Unix, Windows.
.. note::
@@ -1268,24 +1295,21 @@
Change the current working directory to *path*.
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
+ This function can support :ref:`working on a file descriptor <path_fd>`. The
+ descriptor must refer to an opened directory, not an open file.
Availability: Unix, Windows.
.. versionadded:: 3.3
Added support for specifying *path* as a file descriptor
- on some platforms, and the *dir_fd*, *effective_ids*, and
- *follow_symlinks* parameters.
+ on some platforms.
.. function:: fchdir(fd)
Change the current working directory to the directory represented by the file
- descriptor *fd*. The descriptor must refer to an opened directory, not an open
- file. Equivalent to ``os.chdir(fd)``.
+ descriptor *fd*. The descriptor must refer to an opened directory, not an
+ open file. Equivalent to ``os.chdir(fd)``.
Availability: Unix.
@@ -1322,12 +1346,7 @@
* :data:`stat.SF_NOUNLINK`
* :data:`stat.SF_SNAPSHOT`
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`follow_symlinks` will examine the symbolic link itself
- instead of the file the link points to. *follow_symlinks* may not be
- supported on your platform; you can check whether or not it is available
- using :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
+ This function can support :ref:`not following symlinks <follow_symlinks>`.
Availability: Unix.
@@ -1367,37 +1386,17 @@
* :data:`stat.S_IWOTH`
* :data:`stat.S_IXOTH`
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`chmod` will examine the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
-
- It is an error to use *dir_fd* or *follow_symlinks* when specifying
- *path* as an open file descriptor.
+ This function can support :ref:`specifying a file descriptor <path_fd>`,
+ :ref:`paths relative to directory descriptors <dir_fd>` and :ref:`not
+ following symlinks <follow_symlinks>`.
Availability: Unix, Windows.
.. note::
- Although Windows supports :func:`chmod`, you can only set the file's read-only
- flag with it (via the ``stat.S_IWRITE`` and ``stat.S_IREAD``
- constants or a corresponding integer value). All other bits are
- ignored.
+ Although Windows supports :func:`chmod`, you can only set the file's
+ read-only flag with it (via the ``stat.S_IWRITE`` and ``stat.S_IREAD``
+ constants or a corresponding integer value). All other bits are ignored.
.. versionadded:: 3.3
Added support for specifying *path* as an open file descriptor,
@@ -1406,31 +1405,12 @@
.. function:: chown(path, uid, gid, *, dir_fd=None, follow_symlinks=True)
- Change the owner and group id of *path* to the numeric *uid* and *gid*. To leave
- one of the ids unchanged, set it to -1.
-
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`chown` will examine the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
-
- It is an error to use *dir_fd* or *follow_symlinks* when specifying
- *path* as an open file descriptor.
+ Change the owner and group id of *path* to the numeric *uid* and *gid*. To
+ leave one of the ids unchanged, set it to -1.
+
+ This function can support :ref:`specifying a file descriptor <path_fd>`,
+ :ref:`paths relative to directory descriptors <dir_fd>` and :ref:`not
+ following symlinks <follow_symlinks>`.
See :func:`shutil.chown` for a higher-level function that accepts names in
addition to numeric ids.
@@ -1442,55 +1422,290 @@
and the *dir_fd* and *follow_symlinks* arguments.
-.. function:: getxattr(path, attribute, *, follow_symlinks=True)
-
- Return the value of the extended filesystem attribute *attribute* for
- *path*. *attribute* can be bytes or str. If it is str, it is encoded
- with the filesystem encoding.
-
- *path* may be specified as either a string or an open file descriptor.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`setxattr` will examine the symbolic link itself
- instead of the file the link points to. It is an error to use
- *follow_symlinks* when specifying *path* as an open file descriptor.
-
- Availability: Linux
+.. function:: lchflags(path, flags)
+
+ Set the flags of *path* to the numeric *flags*, like :func:`chflags`, but do
+ not follow symbolic links. Equivalent to ``os.chflags(path, flags,
+ follow_symlinks=False)``.
+
+ Availability: Unix.
+
+
+.. function:: lchmod(path, mode)
+
+ Change the mode of *path* to the numeric *mode*. If path is a symlink, this
+ affects the symlink rather than the target. See the docs for :func:`chmod`
+ for possible values of *mode*. Equivalent to ``os.chmod(path, mode,
+ follow_symlinks=False)``.
+
+ Availability: Unix.
+
+
+.. function:: lchown(path, uid, gid)
+
+ Change the owner and group id of *path* to the numeric *uid* and *gid*. This
+ function will not follow symbolic links. Equivalent to ``os.chown(path, uid,
+ gid, follow_symlinks=False)``.
+
+ Availability: Unix.
+
+
+.. function:: link(src, dst, *, src_dir_fd=None, dst_dir_fd=None, follow_symlinks=True)
+
+ Create a hard link pointing to *src* named *dst*.
+
+ If either *src_dir_fd* or *dst_dir_fd* is not ``None``, it should be a file
+ descriptor referring to a directory, and the corresponding path (*src* or
+ *dst*) should be relative; that path will then be relative to that directory.
+ (If *src* is absolute, *src_dir_fd* is ignored; the same goes for *dst* and
+ *dst_dir_fd*.) *src_dir_fd* and *dst_dir_fd* may not be supported on your
+ platform; you can check whether or not they are available using
+ :data:`os.supports_dir_fd`. If they are unavailable, using either will raise
+ a :exc:`NotImplementedError`.
+
+ This function can also support :ref:`not following symlinks
+ <follow_symlinks>`.
+
+ Availability: Unix, Windows.
+
+ .. versionchanged:: 3.2
+ Added Windows support.
.. versionadded:: 3.3
-
-
-.. function:: lchflags(path, flags)
-
- Set the flags of *path* to the numeric *flags*, like :func:`chflags`, but do not
- follow symbolic links.
- Equivalent to ``os.chflags(path, flags, follow_symlinks=False)``.
+ Added the *src_dir_fd*, *dst_dir_fd*, and *follow_symlinks* arguments.
+
+
+.. function:: listdir(path='.')
+
+ Return a list containing the names of the entries in the directory given by
+ *path* (default: ``'.'``). The list is in arbitrary order. It does not
+ include the special entries ``'.'`` and ``'..'`` even if they are present in
+ the directory.
+
+ This function can be called with a bytes or string argument, and returns
+ filenames of the same datatype.
+
+ This function can also support :ref:`specifying an open file descriptor
+ <path_fd>` (referring to a directory).
+
+ Availability: Unix, Windows.
+
+ .. versionchanged:: 3.2
+ The *path* parameter became optional.
+
+ .. versionadded:: 3.3
+ Added support for specifying an open file descriptor for *path*.
+
+
+.. function:: lstat(path, *, dir_fd=None)
+
+ Perform the equivalent of an :c:func:`lstat` system call on the given path.
+ Similar to :func:`~os.stat`, but does not follow symbolic links. On
+ platforms that do not support symbolic links, this is an alias for
+ :func:`~os.stat`. (Equivalent to ``os.stat(path, follow_symlinks=False)``.)
+
+ This function can also support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ .. versionchanged:: 3.2
+ Added support for Windows 6.0 (Vista) symbolic links.
+
+ .. versionchanged:: 3.3
+ Added the *dir_fd* parameter.
+
+
+.. function:: mkfifo(path, mode=0o666, *, dir_fd=None)
+
+ Create a FIFO (a named pipe) named *path* with numeric mode *mode*.
+ The current umask value is first masked out from the mode.
+
+ This function can also support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ FIFOs are pipes that can be accessed like regular files. FIFOs exist until they
+ are deleted (for example with :func:`os.unlink`). Generally, FIFOs are used as
+ rendezvous between "client" and "server" type processes: the server opens the
+ FIFO for reading, and the client opens it for writing. Note that :func:`mkfifo`
+ doesn't open the FIFO --- it just creates the rendezvous point.
Availability: Unix.
-
-.. function:: lchmod(path, mode)
-
- Change the mode of *path* to the numeric *mode*. If path is a symlink, this
- affects the symlink rather than the target. See the docs for :func:`chmod`
- for possible values of *mode*.
- Equivalent to ``os.chmod(path, mode, follow_symlinks=False)``.
+ .. versionadded:: 3.3
+ The *dir_fd* argument.
+
+
+.. function:: mknod(filename, mode=0o600, device=0, *, dir_fd=None)
+
+ Create a filesystem node (file, device special file or named pipe) named
+ *filename*. *mode* specifies both the permissions to use and the type of node
+ to be created, being combined (bitwise OR) with one of ``stat.S_IFREG``,
+ ``stat.S_IFCHR``, ``stat.S_IFBLK``, and ``stat.S_IFIFO`` (those constants are
+ available in :mod:`stat`). For ``stat.S_IFCHR`` and ``stat.S_IFBLK``,
+ *device* defines the newly created device special file (probably using
+ :func:`os.makedev`), otherwise it is ignored.
+
+ This function can also support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ .. versionadded:: 3.3
+ The *dir_fd* argument.
+
+
+.. function:: major(device)
+
+ Extract the device major number from a raw device number (usually the
+ :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`).
+
+
+.. function:: minor(device)
+
+ Extract the device minor number from a raw device number (usually the
+ :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`).
+
+
+.. function:: makedev(major, minor)
+
+ Compose a raw device number from the major and minor device numbers.
+
+
+.. function:: mkdir(path, mode=0o777, *, dir_fd=None)
+
+ Create a directory named *path* with numeric mode *mode*.
+
+ On some systems, *mode* is ignored. Where it is used, the current umask
+ value is first masked out. If the directory already exists, :exc:`OSError`
+ is raised.
+
+ This function can also support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ It is also possible to create temporary directories; see the
+ :mod:`tempfile` module's :func:`tempfile.mkdtemp` function.
+
+ Availability: Unix, Windows.
+
+ .. versionadded:: 3.3
+ The *dir_fd* argument.
+
+
+.. function:: makedirs(path, mode=0o777, exist_ok=False)
+
+ .. index::
+ single: directory; creating
+ single: UNC paths; and os.makedirs()
+
+ Recursive directory creation function. Like :func:`mkdir`, but makes all
+ intermediate-level directories needed to contain the leaf directory. If
+ the target directory with the same mode as specified already exists,
+ raises an :exc:`OSError` exception if *exist_ok* is False, otherwise no
+ exception is raised. If the directory cannot be created in other cases,
+ raises an :exc:`OSError` exception. The default *mode* is ``0o777`` (octal).
+ On some systems, *mode* is ignored. Where it is used, the current umask
+ value is first masked out.
+
+ .. note::
+
+ :func:`makedirs` will become confused if the path elements to create
+ include :data:`pardir`.
+
+ This function handles UNC paths correctly.
+
+ .. versionadded:: 3.2
+ The *exist_ok* parameter.
+
+
+.. function:: pathconf(path, name)
+
+ Return system configuration information relevant to a named file. *name*
+ specifies the configuration value to retrieve; it may be a string which is the
+ name of a defined system value; these names are specified in a number of
+ standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define
+ additional names as well. The names known to the host operating system are
+ given in the ``pathconf_names`` dictionary. For configuration variables not
+ included in that mapping, passing an integer for *name* is also accepted.
+
+ If *name* is a string and is not known, :exc:`ValueError` is raised. If a
+ specific value for *name* is not supported by the host system, even if it is
+ included in ``pathconf_names``, an :exc:`OSError` is raised with
+ :const:`errno.EINVAL` for the error number.
Availability: Unix.
-.. function:: lchown(path, uid, gid)
-
- Change the owner and group id of *path* to the numeric *uid* and *gid*. This
- function will not follow symbolic links.
- Equivalent to ``os.chown(path, uid, gid, follow_symlinks=False)``.
-
- Availability: Unix.
-
-
-.. function:: link(src, dst, *, src_dir_fd=None, dst_dir_fd=None, follow_symlinks=True)
-
- Create a hard link pointing to *src* named *dst*.
+.. data:: pathconf_names
+
+ Dictionary mapping names accepted by :func:`pathconf` and :func:`fpathconf` to
+ the integer values defined for those names by the host operating system. This
+ can be used to determine the set of names known to the system. Availability:
+ Unix.
+
+
+.. function:: readlink(path, *, dir_fd=None)
+
+ Return a string representing the path to which the symbolic link points. The
+ result may be either an absolute or relative pathname; if it is relative, it
+ may be converted to an absolute pathname using
+ ``os.path.join(os.path.dirname(path), result)``.
+
+ If the *path* is a string object, the result will also be a string object,
+ and the call may raise an UnicodeDecodeError. If the *path* is a bytes
+ object, the result will be a bytes object.
+
+ This function can also support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ Availability: Unix, Windows
+
+ .. versionchanged:: 3.2
+ Added support for Windows 6.0 (Vista) symbolic links.
+
+ .. versionadded:: 3.3
+ The *dir_fd* argument.
+
+
+.. function:: remove(path, *, dir_fd=None)
+
+ Remove (delete) the file *path*. If *path* is a directory, :exc:`OSError` is
+ raised. Use :func:`rmdir` to remove directories.
+
+ This function can support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
+
+ On Windows, attempting to remove a file that is in use causes an exception to
+ be raised; on Unix, the directory entry is removed but the storage allocated
+ to the file is not made available until the original file is no longer in use.
+
+ This function is identical to :func:`unlink`.
+
+ Availability: Unix, Windows.
+
+ .. versionadded:: 3.3
+ The *dir_fd* argument.
+
+
+.. function:: removedirs(path)
+
+ .. index:: single: directory; deleting
+
+ Remove directories recursively. Works like :func:`rmdir` except that, if the
+ leaf directory is successfully removed, :func:`removedirs` tries to
+ successively remove every parent directory mentioned in *path* until an error
+ is raised (which is ignored, because it generally means that a parent directory
+ is not empty). For example, ``os.removedirs('foo/bar/baz')`` will first remove
+ the directory ``'foo/bar/baz'``, and then remove ``'foo/bar'`` and ``'foo'`` if
+ they are empty. Raises :exc:`OSError` if the leaf directory could not be
+ successfully removed.
+
+
+.. function:: rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
+
+ Rename the file or directory *src* to *dst*. If *dst* is a directory,
+ :exc:`OSError` will be raised. On Unix, if *dst* exists and is a file, it will
+ be replaced silently if the user has permission. The operation may fail on some
+ Unix flavors if *src* and *dst* are on different filesystems. If successful,
+ the renaming will be an atomic operation (this is a POSIX requirement). On
+ Windows, if *dst* already exists, :exc:`OSError` will be raised even if it is a
+ file.
If either *src_dir_fd* or *dst_dir_fd* is not ``None``, it should be a
file descriptor referring to a directory, and the corresponding path
@@ -1501,322 +1716,34 @@
you can check whether or not they are available using :data:`os.supports_dir_fd`.
If they are unavailable, using either will raise a :exc:`NotImplementedError`.
- If *follow_symlinks* is ``False``, and the last element of *src* is a
- symbolic link, :func:`link` will use the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
+ If you want cross-platform overwriting of the destination, use :func:`replace`.
Availability: Unix, Windows.
- .. versionchanged:: 3.2
- Added Windows support.
-
.. versionadded:: 3.3
- Added the *src_dir_fd*, *dst_dir_fd*, and *follow_symlinks* arguments.
-
-
-.. function:: listdir(path='.')
-
- Return a list containing the names of the entries in the directory given by
- *path* (default: ``'.'``). The list is in arbitrary order. It does not include the special
- entries ``'.'`` and ``'..'`` even if they are present in the directory.
-
- This function can be called with a bytes or string argument, and returns
- filenames of the same datatype.
-
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
-
- Availability: Unix, Windows.
-
- .. versionchanged:: 3.2
- The *path* parameter became optional.
-
- .. versionadded:: 3.3
- Added support for specifying an open file descriptor for *path*.
-
-.. function:: listxattr(path=None, *, follow_symlinks=True)
-
- Return a list of the extended filesystem attributes on *path*.
- The attributes in the list are represented as strings decoded
- with the filesystem encoding.
-
- *path* may be specified as either ``None``, a string, or an open file
- descriptor. If *path* is ``None``, :func:`listxattr` will examine the
- current directory.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`listxattr` will examine the symbolic link itself
- instead of the file the link points to. It is an error to use
- *follow_symlinks* when specifying *path* as an open file descriptor.
-
- Availability: Linux
-
- .. versionadded:: 3.3
-
-
-.. function:: lstat(path, *, dir_fd=None)
-
- Perform the equivalent of an :c:func:`lstat` system call on the given path.
- Similar to :func:`~os.stat`, but does not follow symbolic links. On
- platforms that do not support symbolic links, this is an alias for
- :func:`~os.stat`. (Equivalent to ``os.stat(path, follow_symlinks=False)``.)
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- .. versionchanged:: 3.2
- Added support for Windows 6.0 (Vista) symbolic links.
-
- .. versionchanged:: 3.3
- Added the *dir_fd* parameter.
-
-
-.. function:: mkfifo(path, mode=0o666, *, dir_fd=None)
-
- Create a FIFO (a named pipe) named *path* with numeric mode *mode*.
- The current umask value is first masked out from the mode.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- FIFOs are pipes that can be accessed like regular files. FIFOs exist until they
- are deleted (for example with :func:`os.unlink`). Generally, FIFOs are used as
- rendezvous between "client" and "server" type processes: the server opens the
- FIFO for reading, and the client opens it for writing. Note that :func:`mkfifo`
- doesn't open the FIFO --- it just creates the rendezvous point.
-
- Availability: Unix.
-
- .. versionadded:: 3.3
- The *dir_fd* argument.
-
-
-.. function:: mknod(filename, mode=0o600, device=0, *, dir_fd=None)
-
- Create a filesystem node (file, device special file or named pipe) named
- *filename*. *mode* specifies both the permissions to use and the type of node
- to be created, being combined (bitwise OR) with one of ``stat.S_IFREG``,
- ``stat.S_IFCHR``, ``stat.S_IFBLK``, and ``stat.S_IFIFO`` (those constants are
- available in :mod:`stat`). For ``stat.S_IFCHR`` and ``stat.S_IFBLK``,
- *device* defines the newly created device special file (probably using
- :func:`os.makedev`), otherwise it is ignored.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- .. versionadded:: 3.3
- The *dir_fd* argument.
-
-
-.. function:: major(device)
-
- Extract the device major number from a raw device number (usually the
- :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`).
-
-
-.. function:: minor(device)
-
- Extract the device minor number from a raw device number (usually the
- :attr:`st_dev` or :attr:`st_rdev` field from :c:type:`stat`).
-
-
-.. function:: makedev(major, minor)
-
- Compose a raw device number from the major and minor device numbers.
-
-
-.. function:: mkdir(path, mode=0o777, *, dir_fd=None)
-
- Create a directory named *path* with numeric mode *mode*.
-
- On some systems, *mode* is ignored. Where it is used, the current
- umask value is first masked out. If the directory already
- exists, :exc:`OSError` is raised.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- It is also possible to create temporary directories; see the
- :mod:`tempfile` module's :func:`tempfile.mkdtemp` function.
-
- Availability: Unix, Windows.
-
- .. versionadded:: 3.3
- The *dir_fd* argument.
-
-
-.. function:: makedirs(path, mode=0o777, exist_ok=False)
-
- .. index::
- single: directory; creating
- single: UNC paths; and os.makedirs()
-
- Recursive directory creation function. Like :func:`mkdir`, but makes all
- intermediate-level directories needed to contain the leaf directory. If
- the target directory with the same mode as specified already exists,
- raises an :exc:`OSError` exception if *exist_ok* is False, otherwise no
- exception is raised. If the directory cannot be created in other cases,
- raises an :exc:`OSError` exception. The default *mode* is ``0o777`` (octal).
- On some systems, *mode* is ignored. Where it is used, the current umask
- value is first masked out.
+ The *src_dir_fd* and *dst_dir_fd* arguments.
+
+
+.. function:: renames(old, new)
+
+ Recursive directory or file renaming function. Works like :func:`rename`, except
+ creation of any intermediate directories needed to make the new pathname good is
+ attempted first. After the rename, directories corresponding to rightmost path
+ segments of the old name will be pruned away using :func:`removedirs`.
.. note::
- :func:`makedirs` will become confused if the path elements to create
- include :data:`pardir`.
-
- This function handles UNC paths correctly.
-
- .. versionadded:: 3.2
- The *exist_ok* parameter.
-
-
-.. function:: pathconf(path, name)
-
- Return system configuration information relevant to a named file. *name*
- specifies the configuration value to retrieve; it may be a string which is the
- name of a defined system value; these names are specified in a number of
- standards (POSIX.1, Unix 95, Unix 98, and others). Some platforms define
- additional names as well. The names known to the host operating system are
- given in the ``pathconf_names`` dictionary. For configuration variables not
- included in that mapping, passing an integer for *name* is also accepted.
-
- If *name* is a string and is not known, :exc:`ValueError` is raised. If a
- specific value for *name* is not supported by the host system, even if it is
- included in ``pathconf_names``, an :exc:`OSError` is raised with
- :const:`errno.EINVAL` for the error number.
-
- Availability: Unix.
-
-
-.. data:: pathconf_names
-
- Dictionary mapping names accepted by :func:`pathconf` and :func:`fpathconf` to
- the integer values defined for those names by the host operating system. This
- can be used to determine the set of names known to the system. Availability:
- Unix.
-
-
-.. function:: readlink(path, *, dir_fd=None)
-
- Return a string representing the path to which the symbolic link points. The
- result may be either an absolute or relative pathname; if it is relative, it may
- be converted to an absolute pathname using ``os.path.join(os.path.dirname(path),
- result)``.
-
- If the *path* is a string object, the result will also be a string object,
- and the call may raise an UnicodeDecodeError. If the *path* is a bytes
- object, the result will be a bytes object.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- Availability: Unix, Windows
-
- .. versionchanged:: 3.2
- Added support for Windows 6.0 (Vista) symbolic links.
-
- .. versionadded:: 3.3
- The *dir_fd* argument.
-
-
-.. function:: remove(path, *, dir_fd=None)
-
- Remove (delete) the file *path*. If *path* is a directory, :exc:`OSError`
- is raised. Use :func:`rmdir` to remove directories.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- On Windows, attempting to remove a file that is in use causes an exception to
- be raised; on Unix, the directory entry is removed but the storage allocated
- to the file is not made available until the original file is no longer in use.
-
- This function is identical to :func:`unlink`.
-
- Availability: Unix, Windows.
-
- .. versionadded:: 3.3
- The *dir_fd* argument.
-
-
-.. function:: removedirs(path)
-
- .. index:: single: directory; deleting
-
- Remove directories recursively. Works like :func:`rmdir` except that, if the
- leaf directory is successfully removed, :func:`removedirs` tries to
- successively remove every parent directory mentioned in *path* until an error
- is raised (which is ignored, because it generally means that a parent directory
- is not empty). For example, ``os.removedirs('foo/bar/baz')`` will first remove
- the directory ``'foo/bar/baz'``, and then remove ``'foo/bar'`` and ``'foo'`` if
- they are empty. Raises :exc:`OSError` if the leaf directory could not be
- successfully removed.
-
-
-.. function:: removexattr(path, attribute, *, follow_symlinks=True)
-
- Removes the extended filesystem attribute *attribute* from *path*.
- *attribute* should be bytes or str. If it is a string, it is encoded
- with the filesystem encoding.
-
- *path* may be specified as either a string or an open file descriptor.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`removexattr` will remove the attribute from the
- symbolic link itself instead of the file the link points to. It is an
- error to use *follow_symlinks* when specifying *path* as an open file
- descriptor.
-
- Availability: Linux
-
- .. versionadded:: 3.3
-
-
-.. function:: rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
+ This function can fail with the new directory structure made if you lack
+ permissions needed to remove the leaf directory or file.
+
+
+.. function:: replace(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
Rename the file or directory *src* to *dst*. If *dst* is a directory,
- :exc:`OSError` will be raised. On Unix, if *dst* exists and is a file, it will
- be replaced silently if the user has permission. The operation may fail on some
- Unix flavors if *src* and *dst* are on different filesystems. If successful,
- the renaming will be an atomic operation (this is a POSIX requirement). On
- Windows, if *dst* already exists, :exc:`OSError` will be raised even if it is a
- file.
+ :exc:`OSError` will be raised. If *dst* exists and is a file, it will
+ be replaced silently if the user has permission. The operation may fail
+ if *src* and *dst* are on different filesystems. If successful,
+ the renaming will be an atomic operation (this is a POSIX requirement).
If either *src_dir_fd* or *dst_dir_fd* is not ``None``, it should be a
file descriptor referring to a directory, and the corresponding path
@@ -1827,44 +1754,6 @@
you can check whether or not they are available using :data:`os.supports_dir_fd`.
If they are unavailable, using either will raise a :exc:`NotImplementedError`.
- If you want cross-platform overwriting of the destination, use :func:`replace`.
-
- Availability: Unix, Windows.
-
- .. versionadded:: 3.3
- The *src_dir_fd* and *dst_dir_fd* arguments.
-
-
-.. function:: renames(old, new)
-
- Recursive directory or file renaming function. Works like :func:`rename`, except
- creation of any intermediate directories needed to make the new pathname good is
- attempted first. After the rename, directories corresponding to rightmost path
- segments of the old name will be pruned away using :func:`removedirs`.
-
- .. note::
-
- This function can fail with the new directory structure made if you lack
- permissions needed to remove the leaf directory or file.
-
-
-.. function:: replace(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
-
- Rename the file or directory *src* to *dst*. If *dst* is a directory,
- :exc:`OSError` will be raised. If *dst* exists and is a file, it will
- be replaced silently if the user has permission. The operation may fail
- if *src* and *dst* are on different filesystems. If successful,
- the renaming will be an atomic operation (this is a POSIX requirement).
-
- If either *src_dir_fd* or *dst_dir_fd* is not ``None``, it should be a
- file descriptor referring to a directory, and the corresponding path
- (*src* or *dst*) should be relative; that path will then be relative to
- that directory. (If *src* is absolute, *src_dir_fd* is ignored; the same
- goes for *dst* and *dst_dir_fd*.)
- *src_dir_fd* and *dst_dir_fd* may not be supported on your platform;
- you can check whether or not they are available using :data:`os.supports_dir_fd`.
- If they are unavailable, using either will raise a :exc:`NotImplementedError`.
-
Availability: Unix, Windows
.. versionadded:: 3.3
@@ -1876,13 +1765,8 @@
empty, otherwise, :exc:`OSError` is raised. In order to remove whole
directory trees, :func:`shutil.rmtree` can be used.
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
+ This function can support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
Availability: Unix, Windows.
@@ -1890,51 +1774,6 @@
The *dir_fd* parameter.
-.. data:: XATTR_SIZE_MAX
-
- The maximum size the value of an extended attribute can be. Currently, this
- is 64 kilobytes on Linux.
-
-
-.. data:: XATTR_CREATE
-
- This is a possible value for the flags argument in :func:`setxattr`. It
- indicates the operation must create an attribute.
-
-
-.. data:: XATTR_REPLACE
-
- This is a possible value for the flags argument in :func:`setxattr`. It
- indicates the operation must replace an existing attribute.
-
-
-.. function:: setxattr(path, attribute, value, flags=0, *, follow_symlinks=True)
-
- Set the extended filesystem attribute *attribute* on *path* to *value*.
- *attribute* must be a bytes or str with no embedded NULs. If it is a str,
- it is encoded with the filesystem encoding. *flags* may be
- :data:`XATTR_REPLACE` or :data:`XATTR_CREATE`. If :data:`XATTR_REPLACE` is
- given and the attribute does not exist, ``EEXISTS`` will be raised.
- If :data:`XATTR_CREATE` is given and the attribute already exists, the
- attribute will not be created and ``ENODATA`` will be raised.
-
- *path* may be specified as either a string or an open file descriptor.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`setxattr` will examine the symbolic link itself
- instead of the file the link points to. It is an error to use
- *follow_symlinks* when specifying *path* as an open file descriptor.
-
- Availability: Linux
-
- .. note::
-
- A bug in Linux kernel versions less than 2.6.39 caused the flags argument
- to be ignored on some filesystems.
-
- .. versionadded:: 3.3
-
-
.. function:: stat(path, *, dir_fd=None, follow_symlinks=True)
Perform the equivalent of a :c:func:`stat` system call on the given path.
@@ -2002,30 +1841,17 @@
If you need the exact timestamps you should always use
:attr:`st_atime_ns`, :attr:`st_mtime_ns`, and :attr:`st_ctime_ns`.
- For backward compatibility, the return value of :func:`~os.stat` is also accessible
- as a tuple of at least 10 integers giving the most important (and portable)
- members of the :c:type:`stat` structure, in the order :attr:`st_mode`,
- :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`, :attr:`st_uid`,
- :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`, :attr:`st_mtime`,
- :attr:`st_ctime`. More items may be added at the end by some implementations.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`stat` will examine the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
-
- It is an error to use *dir_fd* or *follow_symlinks* when specifying
- *path* as an open file descriptor.
+ For backward compatibility, the return value of :func:`~os.stat` is also
+ accessible as a tuple of at least 10 integers giving the most important (and
+ portable) members of the :c:type:`stat` structure, in the order
+ :attr:`st_mode`, :attr:`st_ino`, :attr:`st_dev`, :attr:`st_nlink`,
+ :attr:`st_uid`, :attr:`st_gid`, :attr:`st_size`, :attr:`st_atime`,
+ :attr:`st_mtime`, :attr:`st_ctime`. More items may be added at the end by
+ some implementations.
+
+ This function can support :ref:`specifying an open file descriptor
+ <path_fd>`, :ref:`specifying a file descriptor <path_fd>` and :ref:`not
+ following symlinks <follow_symlinks>`.
.. index:: module: stat
@@ -2094,10 +1920,7 @@
read-only, and if :const:`ST_NOSUID` is set, the semantics of
setuid/setgid bits are disabled or not supported.
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
+ This function can support :ref:`specifying a file descriptor <path_fd>`.
.. versionchanged:: 3.2
The :const:`ST_RDONLY` and :const:`ST_NOSUID` constants were added.
@@ -2132,17 +1955,17 @@
.. data:: supports_effective_ids
An object implementing collections.Set indicating which functions in the
- :mod:`os` permit use of the *effective_id* parameter for :func:`os.access`.
+ :mod:`os` permit use of the *effective_ids* parameter for :func:`os.access`.
If the local platform supports it, the collection will contain
:func:`os.access`, otherwise it will be empty.
- To check whether you can use the *effective_id* parameter for
+ To check whether you can use the *effective_ids* parameter for
:func:`os.access`, use the ``in`` operator on ``supports_dir_fd``, like so::
os.access in os.supports_effective_ids
- Currently *effective_id* only works on UNIX platforms;
- it does not work on Windows.
+ Currently *effective_ids* only works on Unix platforms; it does not work on
+ Windows.
.. versionadded:: 3.3
@@ -2196,13 +2019,8 @@
Symbolic link support was introduced in Windows 6.0 (Vista). :func:`symlink`
will raise a :exc:`NotImplementedError` on Windows versions earlier than 6.0.
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
+ This function can support :ref:`paths relative to directory descriptors
+ <dir_fd>`.
.. note::
@@ -2247,7 +2065,7 @@
.. function:: unlink(path, *, dir_fd=None)
Remove (delete) the file *path*. This function is identical to
- :func:`remove`; the :func:`unlink` name is its traditional Unix
+ :func:`remove`; the ``unlink`` name is its traditional Unix
name. Please see the documentation for :func:`remove` for
further information.
@@ -2287,28 +2105,9 @@
use the *st_atime_ns* and *st_mtime_ns* fields from the :func:`os.stat`
result object with the *ns* parameter to `utime`.
- On some platforms, *path* may also be specified as an open file descriptor.
- This functionality may not be supported on your platform; you can check
- whether or not it is available using :data:`os.supports_fd`. If it is
- unavailable, using it will raise a :exc:`NotImplementedError`.
-
- If *dir_fd* is not ``None``, it should be a file descriptor referring to a
- directory, and *path* should be relative; path will then be relative to
- that directory. (If *path* is absolute, *dir_fd* is ignored.)
- *dir_fd* may not be supported on your platform;
- you can check whether or not it is available using
- :data:`os.supports_dir_fd`. If it is unavailable, using it will raise
- a :exc:`NotImplementedError`.
-
- If *follow_symlinks* is ``False``, and the last element of the path is a
- symbolic link, :func:`utime` will examine the symbolic link itself instead
- of the file the link points to. *follow_symlinks* may not be supported
- on your platform; you can check whether or not it is available using
- :data:`os.supports_follow_symlinks`. If it is unavailable,
- using it will raise a :exc:`NotImplementedError`.
-
- It is an error to use *dir_fd* or *follow_symlinks* when specifying
- *path* as an open file descriptor.
+ This function can support :ref:`specifying a file descriptor <path_fd>`,
+ :ref:`paths relative to directory descriptors <dir_fd>` and :ref:`not
+ following symlinks <follow_symlinks>`.
Availability: Unix, Windows.
@@ -2362,9 +2161,9 @@
.. note::
- Be aware that setting *followlinks* to ``True`` can lead to infinite recursion if a
- link points to a parent directory of itself. :func:`walk` does not keep track of
- the directories it visited already.
+ Be aware that setting *followlinks* to ``True`` can lead to infinite
+ recursion if a link points to a parent directory of itself. :func:`walk`
+ does not keep track of the directories it visited already.
.. note::
@@ -2451,6 +2250,81 @@
.. versionadded:: 3.3
+Linux extended attributes
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 3.3
+
+These functions are all available on Linux only.
+
+.. function:: getxattr(path, attribute, *, follow_symlinks=True)
+
+ Return the value of the extended filesystem attribute *attribute* for
+ *path*. *attribute* can be bytes or str. If it is str, it is encoded
+ with the filesystem encoding.
+
+ This function can support :ref:`specifying a file descriptor <path_fd>` and
+ :ref:`not following symlinks <follow_symlinks>`.
+
+
+.. function:: listxattr(path=None, *, follow_symlinks=True)
+
+ Return a list of the extended filesystem attributes on *path*. The
+ attributes in the list are represented as strings decoded with the filesystem
+ encoding. If *path* is ``None``, :func:`listxattr` will examine the current
+ directory.
+
+ This function can support :ref:`specifying a file descriptor <path_fd>` and
+ :ref:`not following symlinks <follow_symlinks>`.
+
+
+.. function:: removexattr(path, attribute, *, follow_symlinks=True)
+
+ Removes the extended filesystem attribute *attribute* from *path*.
+ *attribute* should be bytes or str. If it is a string, it is encoded
+ with the filesystem encoding.
+
+ This function can support :ref:`specifying a file descriptor <path_fd>` and
+ :ref:`not following symlinks <follow_symlinks>`.
+
+
+.. function:: setxattr(path, attribute, value, flags=0, *, follow_symlinks=True)
+
+ Set the extended filesystem attribute *attribute* on *path* to *value*.
+ *attribute* must be a bytes or str with no embedded NULs. If it is a str,
+ it is encoded with the filesystem encoding. *flags* may be
+ :data:`XATTR_REPLACE` or :data:`XATTR_CREATE`. If :data:`XATTR_REPLACE` is
+ given and the attribute does not exist, ``EEXISTS`` will be raised.
+ If :data:`XATTR_CREATE` is given and the attribute already exists, the
+ attribute will not be created and ``ENODATA`` will be raised.
+
+ This function can support :ref:`specifying a file descriptor <path_fd>` and
+ :ref:`not following symlinks <follow_symlinks>`.
+
+ .. note::
+
+ A bug in Linux kernel versions less than 2.6.39 caused the flags argument
+ to be ignored on some filesystems.
+
+
+.. data:: XATTR_SIZE_MAX
+
+ The maximum size the value of an extended attribute can be. Currently, this
+ is 64 kilobytes on Linux.
+
+
+.. data:: XATTR_CREATE
+
+ This is a possible value for the flags argument in :func:`setxattr`. It
+ indicates the operation must create an attribute.
+
+
+.. data:: XATTR_REPLACE
+
+ This is a possible value for the flags argument in :func:`setxattr`. It
+ indicates the operation must replace an existing attribute.
+
+
.. _os-process:
Process Management
diff --git a/Lib/test/test_random.py b/Lib/test/test_random.py
--- a/Lib/test/test_random.py
+++ b/Lib/test/test_random.py
@@ -34,8 +34,12 @@
self.assertEqual(randseq, self.randomlist(N))
def test_seedargs(self):
+ # Seed value with a negative hash.
+ class MySeed(object):
+ def __hash__(self):
+ return -1729
for arg in [None, 0, 0, 1, 1, -1, -1, 10**20, -(10**20),
- 3.14, 1+2j, 'a', tuple('abc')]:
+ 3.14, 1+2j, 'a', tuple('abc'), MySeed()]:
self.gen.seed(arg)
for arg in [list(range(3)), dict(one=1)]:
self.assertRaises(TypeError, self.gen.seed, arg)
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -55,6 +55,10 @@
Library
-------
+- Speed up _decimal by another 10-15% by caching the thread local context
+ that was last accessed. In the pi benchmark (64-bit platform, prec=9),
+ _decimal is now only 1.5x slower than float.
+
- Remove the packaging module, which is not ready for prime time.
- Issue #15154: Add "dir_fd" parameter to os.rmdir, remove "rmdir"
diff --git a/Modules/_decimal/_decimal.c b/Modules/_decimal/_decimal.c
--- a/Modules/_decimal/_decimal.c
+++ b/Modules/_decimal/_decimal.c
@@ -76,6 +76,9 @@
PyObject *traps;
PyObject *flags;
int capitals;
+#ifndef WITHOUT_THREADS
+ PyThreadState *tstate;
+#endif
} PyDecContextObject;
typedef struct {
@@ -123,6 +126,8 @@
#else
/* Key for thread state dictionary */
static PyObject *tls_context_key = NULL;
+/* Invariant: NULL or the most recently accessed thread local context */
+static PyDecContextObject *cached_context = NULL;
#endif
/* Template for creating new thread contexts, calling Context() without
@@ -1182,6 +1187,9 @@
SdFlagAddr(self->flags) = &ctx->status;
CtxCaps(self) = 1;
+#ifndef WITHOUT_THREADS
+ self->tstate = NULL;
+#endif
return (PyObject *)self;
}
@@ -1189,6 +1197,11 @@
static void
context_dealloc(PyDecContextObject *self)
{
+#ifndef WITHOUT_THREADS
+ if (self == cached_context) {
+ cached_context = NULL;
+ }
+#endif
Py_XDECREF(self->traps);
Py_XDECREF(self->flags);
Py_TYPE(self)->tp_free(self);
@@ -1555,18 +1568,19 @@
}
#else
/*
- * Thread local storage currently has a speed penalty of about 16%.
+ * Thread local storage currently has a speed penalty of about 4%.
* All functions that map Python's arithmetic operators to mpdecimal
* functions have to look up the current context for each and every
* operation.
*/
-/* Return borrowed reference to thread local context. */
+/* Get the context from the thread state dictionary. */
static PyObject *
-current_context(void)
+current_context_from_dict(void)
{
- PyObject *dict = NULL;
- PyObject *tl_context = NULL;
+ PyObject *dict;
+ PyObject *tl_context;
+ PyThreadState *tstate;
dict = PyThreadState_GetDict();
if (dict == NULL) {
@@ -1577,32 +1591,54 @@
tl_context = PyDict_GetItemWithError(dict, tls_context_key);
if (tl_context != NULL) {
- /* We already have a thread local context and
- * return a borrowed reference. */
+ /* We already have a thread local context. */
CONTEXT_CHECK(tl_context);
- return tl_context;
- }
- if (PyErr_Occurred()) {
- return NULL;
- }
-
- /* Otherwise, set up a new thread local context. */
- tl_context = context_copy(default_context_template);
- if (tl_context == NULL) {
- return NULL;
- }
- CTX(tl_context)->status = 0;
-
- if (PyDict_SetItem(dict, tls_context_key, tl_context) < 0) {
+ }
+ else {
+ if (PyErr_Occurred()) {
+ return NULL;
+ }
+
+ /* Set up a new thread local context. */
+ tl_context = context_copy(default_context_template);
+ if (tl_context == NULL) {
+ return NULL;
+ }
+ CTX(tl_context)->status = 0;
+
+ if (PyDict_SetItem(dict, tls_context_key, tl_context) < 0) {
+ Py_DECREF(tl_context);
+ return NULL;
+ }
Py_DECREF(tl_context);
- return NULL;
- }
- Py_DECREF(tl_context);
-
- /* refcount is 1 */
+ }
+
+ /* Cache the context of the current thread, assuming that it
+ * will be accessed several times before a thread switch. */
+ tstate = PyThreadState_GET();
+ if (tstate) {
+ cached_context = (PyDecContextObject *)tl_context;
+ cached_context->tstate = tstate;
+ }
+
+ /* Borrowed reference with refcount==1 */
return tl_context;
}
+/* Return borrowed reference to thread local context. */
+static PyObject *
+current_context(void)
+{
+ PyThreadState *tstate;
+
+ tstate = PyThreadState_GET();
+ if (cached_context && cached_context->tstate == tstate) {
+ return (PyObject *)cached_context;
+ }
+
+ return current_context_from_dict();
+}
+
/* ctxobj := borrowed reference to the current context */
#define CURRENT_CONTEXT(ctxobj) \
ctxobj = current_context(); \
@@ -1664,6 +1700,7 @@
Py_INCREF(v);
}
+ cached_context = NULL;
if (PyDict_SetItem(dict, tls_context_key, v) < 0) {
Py_DECREF(v);
return NULL;
diff --git a/Modules/_randommodule.c b/Modules/_randommodule.c
--- a/Modules/_randommodule.c
+++ b/Modules/_randommodule.c
@@ -228,16 +228,17 @@
Py_INCREF(Py_None);
return Py_None;
}
- /* If the arg is an int or long, use its absolute value; else use
- * the absolute value of its hash code.
+ /* This algorithm relies on the number being unsigned.
+ * So: if the arg is a PyLong, use its absolute value.
+ * Otherwise use its hash value, cast to unsigned.
*/
if (PyLong_Check(arg))
n = PyNumber_Absolute(arg);
else {
- Py_ssize_t hash = PyObject_Hash(arg);
+ Py_hash_t hash = PyObject_Hash(arg);
if (hash == -1)
goto Done;
- n = PyLong_FromSsize_t(hash);
+ n = PyLong_FromSize_t((size_t)hash);
}
if (n == NULL)
goto Done;
--
Repository URL: http://hg.python.org/cpython
More information about the Python-checkins
mailing list