(With a lot of cutting):
Peter Hansen wrote:
> Michael Mossey wrote:
>>Is it inefficient in Python to have something like:
>>myprog.py:
>>from mylib import *
>>from math import *
>>mylib.py
>>from math import *
>>Does this do the work of importing math twice?
I'd say there is an inefficiency, but not so great as file I/O.
> 1. Never optimize prematurely. (Sorry, had to say it. :-)
There is an inefficiency in such designs that can be avoided
only by adopting particular designs. Knowing about inefficiency
is not equivalent to premature optimization. Sorry, I see the
smiley, but some bad designs happen by not knowing the sources
of inefficiency.
> 2. Python doesn't actually import the module ... more than once....
> 3. Try to avoid "from xxxx import *" in all but special cases.
> Check the archives for background on this (keywords
> "from module import" ?) but basically except in cases such as
> (maybe) math and wxPython, it can be a very bad idea. Definitely
> not a good habit to get into.
While I essentially agree with Peter, I thought I'd blather about a
performance cost of the "from xxxx import *" form. I'm sure many
people (including Peter) know this, but a naive reader may see this
pair as saying, "The performance cost of importing should be ignored,
but there is a readability reason to avoid lots of 'import *'s."
On small scales, this is true. In the case of _large_ symbol sets,
there is also a performance issue. With an "import *", you add all
of the public symbols from one module into the symbol dictionary of
another. This has two reasonably large costs: the size of the symbol
table of the importing module can get much larger, and the actual
import statement does work per-symbol when adding symbols to its
symbol table.
In the original example, these issues are completely ignorable
(math has only 29 symbols). If you are talking about something
like wxPython.wx, you are talking about over 2700 symbols, and
the cost may start to be significant.
So, in:
myprog.py:
from mylib import *
from wxPython.wx import *
...
mylib.py:
from wxPython.wx import *
...
mylib.py, when first imported, does 2700+ dictionary writes,
then myprog.py does 2700+ dictionary writes (to copy mylib.py's
symbols), and then another 2700+ dictionary writes (to copy
wxPython.wx's symbols). Still not terrible (8000 dictionary
writes doesn't take all that long), but if you have 32 mylib.py
-like things, you are beginning to talk real work. Also note,
you at least have three modules in memory with >2700 symbols:
wxPython.wx, mylib, and myprog.
In contrast:
myprog.py:
from mylib import a,b,c
from wxPython.wx import *
...
mylib.py:
from wxPython.wx import *
...
mylib.py, when first imported, does 2700+ dictionary writes, then
myprog.py does 3 dictionary writes (to copy mylib.py's a, b, and
c), and then another 2700+dictionary writes (to copy wxPython.wx's
symbols). We cut the dictionary work of all of this importing
to two thirds of what it was before. Not phenomenal, but
(A) we only changed one line in our source, and
(B) if you have a lot of modules, each loading many of the others,
you'll do far better than reduction by a third.
You still have the same-sized dictionaries in the modules.
Even better, (but requiring lots of source changes):
myprog.py:
from mylib import a,b,c
from wxPython import wx
... (using wx.symbol for wx ops and values) ...
mylib.py:
from wxPython import wx
... (using wx.symbol for wx ops and values) ...
mylib.py, when first imported, does a few dictionary writes, then
myprog.py does 3 dictionary writes (to copy mylib.py's a, b, and
c), and then another dictionary write (writing the 'wx' entry).
We cut all of the big the dictionary work from the importing.
Of course, wxPython itself does a bunch of dictionary updates as
well, but we've eliminated _our_ heavy dictionary work. Also,
the size of module dictionaries for mylib and myprog are tiny now.
I'm currently experimenting with a module to provide wxPython
symbols in a relatively easy-to use form. If successful, the
source changes needed above will be relatively minor. I've been
able to hand-convert wxPython's demo programs in about an hour
each.
-Scott David Daniels
Scott.Daniels at Acm.Org