Referring to a module by a string without using eval()

[Peter Otten]

jeanbigboute at gmail.com wrote:

> I am trying to write some recursive code to explore the methods, classes,
> functions, builtins, etc. of a package all the way down the hierarchy.
> 
> 1) Preliminaries
> In [2]: def explore_pkg(pkg):
>    ...:     return dir(pkg)
>    ...:
> 
> In [3]: import numpy as np
> 
> In [4]: l2 = explore_pkg(np.random)
> 
> In [5]: len(l2)
> Out[5]: 72 # np.random has 72 'things' underneath it
> 
> In [6]: l2[0:5]
> Out[6]: ['Lock', 'RandomState', '__RandomState_ctor', '__all__',
> '__builtins__']
> 
> 2) I ultimately need to create inputs to explore_pkg programmatically en
> route to a recursively called function.  The only way I can think of is to
> use strings.  But, passing a string such as 'np.random' into explore_pkg
> correctly returns the methods/... associated with the string and not the
> module np.random
> 
> e.g. explore_pkg('np.random') will NOT do what I want
> 
> explore_pkg(eval('np.random')) does work but I understand eval is
> dangerous and not to be trifled with.

There's no point preventing a person from doing eval("shutil.rmtree('/')'") 
that can type

>>> import shutil
>>> shutil.rmtree("/")

For personal tools I have no qualms using it if it simplifies the resulting 
code significantly. Most of the time it doesn't...

> explore_pkg(getattr(np,'random')) works but if I want to go deeper, I have
> to nest getattrs.

If you don't want to write out that for loop use reduce():

>>> import functools
>>> class A: pass
... 
>>> a = A()
>>> a.b = A()
>>> a.b.c = A()
>>> a.b.c.d = 42
>>> functools.reduce(getattr, "b.c.d".split("."), a)
42

> 
> Question: Is there a solution to this "turn a string into the module it
> represents" problem?  I have a vague feeling decorators might be what I
> need but they've always confused me.
> 
> I have searched extensively but couldn't find anything directly related to
> this.

If you use the original module name rather than the np alias it's easy:

>>> import importlib
>>> importlib.import_module("numpy.random")
<module 'numpy.random' from '/usr/lib/python3/dist-
packages/numpy/random/__init__.py'>

If you want to check the __main__ module first:

>>> import numpy as np
>>> import __main__
>>> import functools
>>> functools.reduce(getattr, "np.random".split("."), __main__)
<module 'numpy.random' from '/usr/lib/python3/dist-
packages/numpy/random/__init__.py'>