[issue36701] module 'urllib' has no attribute 'request'

[Martin Panter]

Martin Panter <vadmium+py at gmail.com> added the comment:

The “urllib” package also contains “urllib.parse”, which is a lot more lightweight than “urllib.request”. In a quick experiment, importing “urllib.request” is more than 2 times slower than importing “urllib.parse” on its own. And importing “urllib” by itself is not much faster, so I guess a lot of the time is unavoidable Python startup, and “urllib.request” is probably many times slower than “urllib.parse”.

The reason for the slowness is the dependencies and initialization. The “urllib.parse” module only imports a few commonly-used modules. On the other hand, importing “urllib.request” imports many heavyweight high-level modules directly and indirectly (email submodules in particular, also things like SSL, multithreading, HTTP client, temporary files). Some of these dependencies also compile lots of regular expressions at import time.

The slowdown can be a problem for things like command-line programs. Just today I found “circusd --help” on a Raspberry Pi took ~5 s to produce output.

The case of “os.path” is different: it isn’t a submodule of “os”. It is just a pointer to “posixpath”, “ntpath”, etc, depending on “os.name”.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36701>
_______________________________________