[Python-checkins] r87270 - python/branches/py3k/Doc/library/urllib.request.rst
antoine.pitrou
python-checkins at python.org
Wed Dec 15 20:07:26 CET 2010
Author: antoine.pitrou
Date: Wed Dec 15 20:07:26 2010
New Revision: 87270
Log:
Move the urllib-inherited API to a distinguished section
Modified:
python/branches/py3k/Doc/library/urllib.request.rst
Modified: python/branches/py3k/Doc/library/urllib.request.rst
==============================================================================
--- python/branches/py3k/Doc/library/urllib.request.rst (original)
+++ python/branches/py3k/Doc/library/urllib.request.rst Wed Dec 15 20:07:26 2010
@@ -104,52 +104,6 @@
member variable to modify its position in the handlers list.
-.. function:: urlretrieve(url, filename=None, reporthook=None, data=None)
-
- Copy a network object denoted by a URL to a local file, if necessary. If the URL
- points to a local file, or a valid cached copy of the object exists, the object
- is not copied. Return a tuple ``(filename, headers)`` where *filename* is the
- local file name under which the object can be found, and *headers* is whatever
- the :meth:`info` method of the object returned by :func:`urlopen` returned (for
- a remote object, possibly cached). Exceptions are the same as for
- :func:`urlopen`.
-
- The second argument, if present, specifies the file location to copy to (if
- absent, the location will be a tempfile with a generated name). The third
- argument, if present, is a hook function that will be called once on
- establishment of the network connection and once after each block read
- thereafter. The hook will be passed three arguments; a count of blocks
- transferred so far, a block size in bytes, and the total size of the file. The
- third argument may be ``-1`` on older FTP servers which do not return a file
- size in response to a retrieval request.
-
- If the *url* uses the :file:`http:` scheme identifier, the optional *data*
- argument may be given to specify a ``POST`` request (normally the request type
- is ``GET``). The *data* argument must in standard
- :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
- function below.
-
- :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that
- the amount of data available was less than the expected amount (which is the
- size reported by a *Content-Length* header). This can occur, for example, when
- the download is interrupted.
-
- The *Content-Length* is treated as a lower bound: if there's more data to read,
- urlretrieve reads more data, but if less data is available, it raises the
- exception.
-
- You can still retrieve the downloaded data in this case, it is stored in the
- :attr:`content` attribute of the exception instance.
-
- If no *Content-Length* header was supplied, urlretrieve can not check the size
- of the data it has downloaded, and just returns it. In this case you just have
- to assume that the download was successful.
-
-.. function:: urlcleanup()
-
- Clear the cache that may have been built up by previous calls to
- :func:`urlretrieve`.
-
.. function:: pathname2url(path)
Convert the pathname *path* from the local syntax for a path to the form used in
@@ -218,116 +172,6 @@
fetching of the image, this should be true.
-.. class:: URLopener(proxies=None, **x509)
-
- Base class for opening and reading URLs. Unless you need to support opening
- objects using schemes other than :file:`http:`, :file:`ftp:`, or :file:`file:`,
- you probably want to use :class:`FancyURLopener`.
-
- By default, the :class:`URLopener` class sends a :mailheader:`User-Agent` header
- of ``urllib/VVV``, where *VVV* is the :mod:`urllib` version number.
- Applications can define their own :mailheader:`User-Agent` header by subclassing
- :class:`URLopener` or :class:`FancyURLopener` and setting the class attribute
- :attr:`version` to an appropriate string value in the subclass definition.
-
- The optional *proxies* parameter should be a dictionary mapping scheme names to
- proxy URLs, where an empty dictionary turns proxies off completely. Its default
- value is ``None``, in which case environmental proxy settings will be used if
- present, as discussed in the definition of :func:`urlopen`, above.
-
- Additional keyword parameters, collected in *x509*, may be used for
- authentication of the client when using the :file:`https:` scheme. The keywords
- *key_file* and *cert_file* are supported to provide an SSL key and certificate;
- both are needed to support client authentication.
-
- :class:`URLopener` objects will raise an :exc:`IOError` exception if the server
- returns an error code.
-
- .. method:: open(fullurl, data=None)
-
- Open *fullurl* using the appropriate protocol. This method sets up cache and
- proxy information, then calls the appropriate open method with its input
- arguments. If the scheme is not recognized, :meth:`open_unknown` is called.
- The *data* argument has the same meaning as the *data* argument of
- :func:`urlopen`.
-
-
- .. method:: open_unknown(fullurl, data=None)
-
- Overridable interface to open unknown URL types.
-
-
- .. method:: retrieve(url, filename=None, reporthook=None, data=None)
-
- Retrieves the contents of *url* and places it in *filename*. The return value
- is a tuple consisting of a local filename and either a
- :class:`email.message.Message` object containing the response headers (for remote
- URLs) or ``None`` (for local URLs). The caller must then open and read the
- contents of *filename*. If *filename* is not given and the URL refers to a
- local file, the input filename is returned. If the URL is non-local and
- *filename* is not given, the filename is the output of :func:`tempfile.mktemp`
- with a suffix that matches the suffix of the last path component of the input
- URL. If *reporthook* is given, it must be a function accepting three numeric
- parameters. It will be called after each chunk of data is read from the
- network. *reporthook* is ignored for local URLs.
-
- If the *url* uses the :file:`http:` scheme identifier, the optional *data*
- argument may be given to specify a ``POST`` request (normally the request type
- is ``GET``). The *data* argument must in standard
- :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
- function below.
-
-
- .. attribute:: version
-
- Variable that specifies the user agent of the opener object. To get
- :mod:`urllib` to tell servers that it is a particular user agent, set this in a
- subclass as a class variable or in the constructor before calling the base
- constructor.
-
-
-.. class:: FancyURLopener(...)
-
- :class:`FancyURLopener` subclasses :class:`URLopener` providing default handling
- for the following HTTP response codes: 301, 302, 303, 307 and 401. For the 30x
- response codes listed above, the :mailheader:`Location` header is used to fetch
- the actual URL. For 401 response codes (authentication required), basic HTTP
- authentication is performed. For the 30x response codes, recursion is bounded
- by the value of the *maxtries* attribute, which defaults to 10.
-
- For all other response codes, the method :meth:`http_error_default` is called
- which you can override in subclasses to handle the error appropriately.
-
- .. note::
-
- According to the letter of :rfc:`2616`, 301 and 302 responses to POST requests
- must not be automatically redirected without confirmation by the user. In
- reality, browsers do allow automatic redirection of these responses, changing
- the POST to a GET, and :mod:`urllib` reproduces this behaviour.
-
- The parameters to the constructor are the same as those for :class:`URLopener`.
-
- .. note::
-
- When performing basic authentication, a :class:`FancyURLopener` instance calls
- its :meth:`prompt_user_passwd` method. The default implementation asks the
- users for the required information on the controlling terminal. A subclass may
- override this method to support more appropriate behavior if needed.
-
- The :class:`FancyURLopener` class offers one additional method that should be
- overloaded to provide the appropriate behavior:
-
- .. method:: prompt_user_passwd(host, realm)
-
- Return information needed to authenticate the user at the given host in the
- specified security realm. The return value should be a tuple, ``(user,
- password)``, which can be used for basic authentication.
-
- The implementation prompts for this information on the terminal; an application
- should override this method to use an appropriate interaction model in the local
- environment.
-
-
.. class:: OpenerDirector()
The :class:`OpenerDirector` class opens URLs via :class:`BaseHandler`\ s chained
@@ -1220,6 +1064,170 @@
>>> f.read().decode('utf-8')
+Legacy interface
+----------------
+
+The following functions and classes are ported from the Python 2 module
+``urllib`` (as opposed to ``urllib2``). They might become deprecated at
+some point in the future.
+
+
+.. function:: urlretrieve(url, filename=None, reporthook=None, data=None)
+
+ Copy a network object denoted by a URL to a local file, if necessary. If the URL
+ points to a local file, or a valid cached copy of the object exists, the object
+ is not copied. Return a tuple ``(filename, headers)`` where *filename* is the
+ local file name under which the object can be found, and *headers* is whatever
+ the :meth:`info` method of the object returned by :func:`urlopen` returned (for
+ a remote object, possibly cached). Exceptions are the same as for
+ :func:`urlopen`.
+
+ The second argument, if present, specifies the file location to copy to (if
+ absent, the location will be a tempfile with a generated name). The third
+ argument, if present, is a hook function that will be called once on
+ establishment of the network connection and once after each block read
+ thereafter. The hook will be passed three arguments; a count of blocks
+ transferred so far, a block size in bytes, and the total size of the file. The
+ third argument may be ``-1`` on older FTP servers which do not return a file
+ size in response to a retrieval request.
+
+ If the *url* uses the :file:`http:` scheme identifier, the optional *data*
+ argument may be given to specify a ``POST`` request (normally the request type
+ is ``GET``). The *data* argument must in standard
+ :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
+ function below.
+
+ :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that
+ the amount of data available was less than the expected amount (which is the
+ size reported by a *Content-Length* header). This can occur, for example, when
+ the download is interrupted.
+
+ The *Content-Length* is treated as a lower bound: if there's more data to read,
+ urlretrieve reads more data, but if less data is available, it raises the
+ exception.
+
+ You can still retrieve the downloaded data in this case, it is stored in the
+ :attr:`content` attribute of the exception instance.
+
+ If no *Content-Length* header was supplied, urlretrieve can not check the size
+ of the data it has downloaded, and just returns it. In this case you just have
+ to assume that the download was successful.
+
+.. function:: urlcleanup()
+
+ Clear the cache that may have been built up by previous calls to
+ :func:`urlretrieve`.
+
+.. class:: URLopener(proxies=None, **x509)
+
+ Base class for opening and reading URLs. Unless you need to support opening
+ objects using schemes other than :file:`http:`, :file:`ftp:`, or :file:`file:`,
+ you probably want to use :class:`FancyURLopener`.
+
+ By default, the :class:`URLopener` class sends a :mailheader:`User-Agent` header
+ of ``urllib/VVV``, where *VVV* is the :mod:`urllib` version number.
+ Applications can define their own :mailheader:`User-Agent` header by subclassing
+ :class:`URLopener` or :class:`FancyURLopener` and setting the class attribute
+ :attr:`version` to an appropriate string value in the subclass definition.
+
+ The optional *proxies* parameter should be a dictionary mapping scheme names to
+ proxy URLs, where an empty dictionary turns proxies off completely. Its default
+ value is ``None``, in which case environmental proxy settings will be used if
+ present, as discussed in the definition of :func:`urlopen`, above.
+
+ Additional keyword parameters, collected in *x509*, may be used for
+ authentication of the client when using the :file:`https:` scheme. The keywords
+ *key_file* and *cert_file* are supported to provide an SSL key and certificate;
+ both are needed to support client authentication.
+
+ :class:`URLopener` objects will raise an :exc:`IOError` exception if the server
+ returns an error code.
+
+ .. method:: open(fullurl, data=None)
+
+ Open *fullurl* using the appropriate protocol. This method sets up cache and
+ proxy information, then calls the appropriate open method with its input
+ arguments. If the scheme is not recognized, :meth:`open_unknown` is called.
+ The *data* argument has the same meaning as the *data* argument of
+ :func:`urlopen`.
+
+
+ .. method:: open_unknown(fullurl, data=None)
+
+ Overridable interface to open unknown URL types.
+
+
+ .. method:: retrieve(url, filename=None, reporthook=None, data=None)
+
+ Retrieves the contents of *url* and places it in *filename*. The return value
+ is a tuple consisting of a local filename and either a
+ :class:`email.message.Message` object containing the response headers (for remote
+ URLs) or ``None`` (for local URLs). The caller must then open and read the
+ contents of *filename*. If *filename* is not given and the URL refers to a
+ local file, the input filename is returned. If the URL is non-local and
+ *filename* is not given, the filename is the output of :func:`tempfile.mktemp`
+ with a suffix that matches the suffix of the last path component of the input
+ URL. If *reporthook* is given, it must be a function accepting three numeric
+ parameters. It will be called after each chunk of data is read from the
+ network. *reporthook* is ignored for local URLs.
+
+ If the *url* uses the :file:`http:` scheme identifier, the optional *data*
+ argument may be given to specify a ``POST`` request (normally the request type
+ is ``GET``). The *data* argument must in standard
+ :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
+ function below.
+
+
+ .. attribute:: version
+
+ Variable that specifies the user agent of the opener object. To get
+ :mod:`urllib` to tell servers that it is a particular user agent, set this in a
+ subclass as a class variable or in the constructor before calling the base
+ constructor.
+
+
+.. class:: FancyURLopener(...)
+
+ :class:`FancyURLopener` subclasses :class:`URLopener` providing default handling
+ for the following HTTP response codes: 301, 302, 303, 307 and 401. For the 30x
+ response codes listed above, the :mailheader:`Location` header is used to fetch
+ the actual URL. For 401 response codes (authentication required), basic HTTP
+ authentication is performed. For the 30x response codes, recursion is bounded
+ by the value of the *maxtries* attribute, which defaults to 10.
+
+ For all other response codes, the method :meth:`http_error_default` is called
+ which you can override in subclasses to handle the error appropriately.
+
+ .. note::
+
+ According to the letter of :rfc:`2616`, 301 and 302 responses to POST requests
+ must not be automatically redirected without confirmation by the user. In
+ reality, browsers do allow automatic redirection of these responses, changing
+ the POST to a GET, and :mod:`urllib` reproduces this behaviour.
+
+ The parameters to the constructor are the same as those for :class:`URLopener`.
+
+ .. note::
+
+ When performing basic authentication, a :class:`FancyURLopener` instance calls
+ its :meth:`prompt_user_passwd` method. The default implementation asks the
+ users for the required information on the controlling terminal. A subclass may
+ override this method to support more appropriate behavior if needed.
+
+ The :class:`FancyURLopener` class offers one additional method that should be
+ overloaded to provide the appropriate behavior:
+
+ .. method:: prompt_user_passwd(host, realm)
+
+ Return information needed to authenticate the user at the given host in the
+ specified security realm. The return value should be a tuple, ``(user,
+ password)``, which can be used for basic authentication.
+
+ The implementation prompts for this information on the terminal; an application
+ should override this method to use an appropriate interaction model in the local
+ environment.
+
+
:mod:`urllib.request` Restrictions
----------------------------------
More information about the Python-checkins
mailing list