Friday Filosofical Finking: Import protections

[Cameron Simpson]

On 18Apr2019 16:05, DL Neil <PythonList at DancesWithMice.info> wrote:
>On 18/04/19 8:45 AM, MRAB wrote:
>>On 2019-04-17 21:20, DL Neil wrote:
>>>Do you bother with exception handling for import statements?
>>>
>>>Can we assume that if such a catastrophic error occurs, it is quite
>>>acceptable for the code to fall-over in a tumbling-fumble?
>
>>[snip]
>>Catch only what you (well, the script) can fix.
>>
>>If it needs numpy, but can't import numpy, then when can it do? 
>>Might as well just let it fail.
>>
>>I suppose an alternative might be to try to download and install 
>>numpy and then retry, but what if it can't be downloaded, or the 
>>installation fails?
>>
>>No, you have to give up at some point. It's better just to report 
>>the problem and leave it to the user to fix it.
>
>Ah, but is that not the point - the user cannot fix it (and neither 
>should he be allowed to do so: see later comments about Operations 
>functions).

I'm missing something here. To me there are usually 2 scenarios where a 
failed import obstructs the user:

- the user is someone like you or I - a person using a programme from 
  source they're fetched - trying to run some programme with a module 
  dependency; here we can reasonably assuming some responsibility for 
  obtaining the missing module; ideally the place we got the programme 
  from might have some pointers

- the user is running some kind of packaged app. With python that tends 
  to be either something obtained via pip from PyPI or a downloaded 
  package, such as a bundle Mac .app or a Windows .msi file or a package 
  from a linux distro. Here the responsibility is with the person who 
  made the package/bundle.

In the latter case the source _should_ have resolvable dependencies. For 
example pip packages can name the modules they require and pip itself 
will fetch those in turn. Bundles apps (.app, .msi etc) tend to include 
the modules within the bundle. Linux distro packages should include 
dependencies as well, and the package programme should fetch those.

All of these should fulfil the requirements at programme install time, 
not at programme run time.

Do you have a third scenario you could detail?

>What the user faces (unless we more properly handle the exception) is:
>
>import davids_brains
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>ModuleNotFoundError: No module named 'davids_brains'
>
>You and I will think this fine - and 'here' on the list will even beg 
>OPs to give us this much data.
>
>However, despite users who have been known to make other comments 
>about my (?lack of) brains, they will tend to offer such helpful advice 
>as: "we need the missing module"...!

I think this case is usually the former of the 2 above: the user has 
fetched from somewhere random. That somewhere should at least identify 
what is required, even if the user needs to do some further legwork 
themselves.

I think my stance here is that it is the person installing the 
app/module who needs to sort this, because they know the context in 
which the app/module is being installed. Do we use pip to get stuff? Do 
we use the OS package manager to get stuff? Are we in a managed 
environment where we have to ask the local sysadmins to do this?

The point is that the app/module can't know how to obtain missing 
components. If they didn't come with it, there are many unknowns about 
the environment and plenty of legitimate circumstances where an 
automatic install is infeasible or actually undesirable.

I think that _most_ ImportErrors should not be caught unless the 
recovery is extremely well defined.

While I'm not a fan of the "import settings" scenario (executable 
settings files? scary), I do catch ImportErrors in several places in my 
personal kit. Like Chris I think this is only appropriate where there's 
a very well defined coping mechanism, in particular _not_ assuming one 
can fetch more software.

So:

Python 2/3 imports: personally I try the python 3 import and fall back 
to the python 2 if that fails. But if the python 2 import fails, _that_ 
import error gets out. No "install" style recovery.

Optional modules: I've got a module which supports several possible 
index backends. So I've got some classes along these lines:

  class LMDBIndex(_Index):
    [...]
    @classmethod
    def is_supported(cls):
      ''' Test whether this index class is supported by the Python environment.
      '''
      try:
        import lmdb
      except ImportError:
        return False
      return True

and some code which looks for a working index class:

    for indexname, indexclass in indexclasses:
      if not indexclass.is_supported():
        continue
      ... some other checks ...
      return indexclass
    ...
    raise ValueError(
        "no supported index classes available: tried %r"
        % (indexclasses,))

So: no recovery or autoinstall. It just chooses the first (== most 
preferred) index class and returns it, and raises an exception if none 
is available.

Cheers,
Cameron Simpson <cs at cskk.id.au>