[Python-ideas] Defer Statement

[Nathaniel Smith]

On Sun, Jun 4, 2017 at 12:23 AM, Lucas Wiman <lucas.wiman at gmail.com> wrote:
> I agree that the stated use cases are better handled with ExitStack.  One
> area where `defer` might be useful is in lazy-evaluating global constants.
> For example in a genomics library used at my work, one module involves
> compiling a large number of regular expressions, and setting them as global
> contants in the module, like:
>
> FOO1_re = re.compile(r'...')
> FOO_TO_BAR_re = {foo: complicated_computation_of_regex(foo) for foo in
> LONG_LIST_OF_THINGS}
> ...
>
> This utility module is imported in a lot of places in the codebase, which
> meant that importing almost anything from our codebase involved precompiling
> all these regular expressions, which took around 500ms to to run the
> anything (the test runner, manually testing code in the shell, etc.) It
> would be ideal to only do these computations if/when they are needed. This
> is a more general issue than this specific example, e.g. for libraries which
> parse large data sources like pycountry (see my PR for one possible ugly
> solution using proxy objects; the author instead went with the simpler, less
> general solution of manually deciding when the data is needed). See also
> django.utils.functional.lazy, which is used extensively in the framework.
>
> A statement like: `defer FOO = lengthy_computation_of_foo()` which deferred
> the lengthy computation until it is used for something would be useful to
> allow easily fixing these issues without writing ugly hacks like proxy
> objects or refactoring code into high-overhead cached properties or the
> like.

I think in general I'd recommend making the API for accessing these
things be a function call interface, so that it's obvious to the
caller that some expensive computation might be going on. But if
you're stuck with an attribute-lookup based interface, then you can
use a __getattr__ hook to compute them the first time they're
accessed:

class LazyConstants:
    def __getattr__(self, name):
        value = compute_value_for(name)
        setattr(self, name, value)
        return value

__getattr__ is only called as a fallback, so by setting the computed
value on the object we make any future attribute lookups just as cheap
as they would be otherwise.

You can get this behavior onto a module object by doing
"sys.modules[__name__] = Constants()" inside the module body, or by
using a <del>hack</del> elegant bit of code like
https://github.com/njsmith/metamodule/ (mostly the latter would only
be preferred if you have a bunch of other attributes exported from
this same module and trying to move all of them onto the LazyConstants
object would be difficult).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org