help on "from deen import *" vs. "import deen"

[Steve D'Aprano]

On Wed, 16 Nov 2016 09:16 am, Erik wrote:

> On 15/11/16 14:43, Michael Torrie wrote:
>> As you've been told several times, if you "import deen" then you can
>> place a new object into the deen namespace using something like:
>>
>> deen.foo=bar
>>
>> Importing everything from an imported module into the current module's
>> namespace is not the best idea
> 
> But "from foo import *" is not importing "everything". It's the opposite
> - it's importing everything _that the module wants to explicitly expose_
> (i.e., a subset of everything ;)).

I think you're skipping Michael's point and making a pedantic comment that
isn't really relevant and, to be even more pedantic, is wrong.

The gory details of what precisely gets imported by an import star is
probably not relevant to a beginner still struggling with understanding
fundamental scoping issues.

I'm also sure that Michael didn't *literally* mean "everything", since he's
been around a while and is probably aware that private names with a single
leading underscore won't be imported.

And your comment is (pedantically) wrong because it isn't always the case
that "import *" only imports things that the module explicitly exposes for
importing. By default, it imports everything which isn't implicitly
excluded. Modules don't pre-define an __all__ variable, hence modules have
to opt out of allowing imports of all non-private names rather than opt in.

I'm sure this is all important for intermediate-level Python programmers,
but I just hope we're not confusing the OP.


> It *used* to be everything in the module's namespace, but then the
> possibly misnamed "__all__" magic variable made it a subset of whatever
> the module's author wanted to expose.

It isn't misnamed: it's intended as a shorter version of "all public names".
The intent being, module authors list ALL their module's public names in
__all__ in order to control what import star does, without relying purely
on the implicit _private name convention.


> However, "import foo" _does_ import "everything", 

No, that's a misuse of terminology. It only imports a single thing: `foo`.


You can test this yourself by comparing the local namespace before and after
a single import: only one more name is added to your namespace.

py> before = after = None
py> before = set(locals().keys())
py> assert 'cmath' not in before
py> import cmath
py> after = set(locals().keys())
py> after - before
{'cmath'}


The conclusion is obvious: importing a single name imports only a single
object into the current namespace. (Well, what else did you expect? *wink*)

What you are talking about is something different: importing a module gives
you access to all the module's attributes. Well of course it does. If you
import a single class from a module, having access to the class gives you
access to all the class attributes. Importing a single function gives you
access to the function's attributes. Importing any object at all gives you
access to that object's attributes. Modules are no different.


> and also gives the 
> importer the power to re-bind names within that module's namespace too
> (which is the crux of the original question). This is not usually a good
> thing unless the importing module is incestuous with the imported module.

With great power comes great responsibility... 

Monkey-patching modules is a powerful and risky technique. No *library*
should monkey-patch another library (unless they're designed to work
together), since you cannot tell if you're messing with something a third
library relies on. But its acceptable for an application to take over
control of a library and monkey-patch it, since there should only ever be a
single application running at once.

Perhaps only *borderline* acceptable, and maybe not something many people
are willing to do in production systems, but its certainly something we do
in (for example) testing. What is a test suite but a stand-alone
application? And test suites will often monkey-patch libraries, insert
mocks and stubs and otherwise manipulate the library.

At least we're not Ruby :-)

http://www.virtuouscode.com/2008/02/23/why-monkeypatching-is-destroying-ruby/


> So this brings up another point - not sure if it has been made before:
> should a module have a way of *not* allowing an importer to re-bind its
> local bindings?

Certainly not! Well, maybe. Yes.

Never mind this "the importer" bit, that's just a special case of a more
general problem: Python has no way of enforcing constants.

This has been argued about many times. Some people want a way of getting
real constants which cannot be modified, or at least cannot be re-bound to
a new value. Others think that the status quo ("if you don't want to
re-bind a constant, then don't re-bind it") is satisfactory.

I'd prefer to have a way to protect against *accidental* re-bindings:

const x = 1.2345
# later:
x += 1  # raise an exception

while still allowing some mechanism for *deliberate* monkey-patching, say:

vars()['x'] = 2.2345  # bypass const mechanism


Possibly something along the lines of property for modules might work. There
are tricks and hacks to get something along these lines already, usually
involving a module replacing itself with the instance of some other class,
but none of them are exactly easy, straightforward or obvious.



> For example, something like a "__bindable__" variable such that all
> names not included may not be bound by any other module? 

You're basically asking for a special case of "protected" and "private" for
classes, except you only want it to apply to modules. Seems weird.

In any case, the usual argument against these things (including the "const"
idea) is that We're All Adults Here. Developers of languages with strong
protections of this sort invariably end up spending large amounts of time
trying to bypass those features, which suggests that they're often
anti-features, or at least too easy to abuse.

Python takes a simpler approach: simple naming conventions (ALLCAPS for
constants, leading _underscore for private names) and trust the caller to
follow the convention.

> And/or 
> something like an "__export__" variable that says what gets exposed to a
> simple "import foo" (sometimes one might want to "import foo" and
> sometimes "from foo import *" for namespace reasons, but why should
> those two statements import different things)?

Because that's what they are designed to do.

`import foo` imports the module foo, that is all. (To be pedantic: it is
*nominally* a module. By design, it could be any object at all.)

`from foo import *` imports all the visible public attributes of foo.

They do completely different things, equivalent to something similar to:

five = int('5')


versus:


_tmp = int('5')
for name in dir(_tmp):
    if not name.startswith('_'):
        locals()[name] = getattr(_tmp, name)
del _tmp



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.