[Speed] Speed Digest, Vol 36, Issue 1

[Bhavishya]

Hello,
I have added the "lazy_import"
<https://github.com/bhavishyagopesh/gsoc_2017/blob/master/python_startup_time/lazy_loader.py>
function but still working on adding  it  implicitly(To ensure that at
startup it is actually  used.)

Thanks Antoine for the suggestion

Regards,
Bhavishya

On Sat, Jun 24, 2017 at 9:30 PM, <speed-request at python.org> wrote:

> Send Speed mailing list submissions to
>         speed at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/speed
> or, via email, send a message with subject or body 'help' to
>         speed-request at python.org
>
> You can reach the person managing the list at
>         speed-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Speed digest..."
>
>
> Today's Topics:
>
>    1. Re: Lazy-loading to decrease python_startup time (Brett Cannon)
>    2. Re: Lazy-loading to decrease python_startup time (Antoine Pitrou)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 23 Jun 2017 23:03:57 +0000
> From: Brett Cannon <brett at python.org>
> To: Bhavishya <bhavishyagopesh at gmail.com>, speed at python.org,  Ramya
>         Meruva <meruvaramya116 at gmail.com>, Victor Stinner
>         <victor.stinner at gmail.com>
> Subject: Re: [Speed] Lazy-loading to decrease python_startup time
> Message-ID:
>         <CAP1=2W41_xTDt7gkuUWnmYA=+Bz9Ox-XT-pAjtXkDGg4TDrY5Q at mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> On Fri, 23 Jun 2017 at 12:17 Bhavishya <bhavishyagopesh at gmail.com> wrote:
>
> > As suggested, I'd like to discuss if lazy-loading is an option for
> > improving python-startup time.And if could be done inside the scope of a
> > GSoc project.
> >
>
> It's a possibility and it could be done in the scope of a GSoC project
> easily. Basically what would be needed is an importlib.util.lazy_import()
> function which does mostly what is outlined in
> https://docs.python.org/3/library/importlib.html#approximating-importlib-
> import-module
> but
> where the proper lazy loader is set on the spec object as an extra step.
> Then every module that is used during startup would use
> importlib.util.lazy_import() for importing instead of the normal import
> statement. What this would do is help guarantee that all modules that are
> identified as part of startup never import a module needlessly as the lazy
> loader would simply postpone the load until necessary. This would also
> allow for pulling out local imports that are currently done in modules that
> are a part of startup and make them global so they are more visible.
>
> But I have no idea if this will actually speed things up. :) At worst it
> would slow things down ever so slightly due to the extra overhead of lazy
> loading for things that are known to be needed already during startup. At
> best, though, is we accidentally discover modules that are being imported
> needlessly at startup as well as not having to hide imports in functions
> for performance. This fact that it may not be worth it is why I haven't
> bothered to try it out yet.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://mail.python.org/pipermail/speed/attachments/
> 20170623/ed7416e2/attachment-0001.html>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 24 Jun 2017 10:59:49 +0200
> From: Antoine Pitrou <solipsis at pitrou.net>
> To: speed at python.org
> Subject: Re: [Speed] Lazy-loading to decrease python_startup time
> Message-ID: <20170624105949.62590bbf at fsol>
> Content-Type: text/plain; charset=US-ASCII
>
> On Fri, 23 Jun 2017 23:03:57 +0000
> Brett Cannon <brett at python.org> wrote:
> > On Fri, 23 Jun 2017 at 12:17 Bhavishya <bhavishyagopesh at gmail.com>
> wrote:
> >
> > > As suggested, I'd like to discuss if lazy-loading is an option for
> > > improving python-startup time.And if could be done inside the scope of
> a
> > > GSoc project.
> > >
> >
> > It's a possibility and it could be done in the scope of a GSoC project
> > easily. Basically what would be needed is an importlib.util.lazy_import()
> > function which does mostly what is outlined in
> > https://docs.python.org/3/library/importlib.html#
> approximating-importlib-import-module
> > but
> > where the proper lazy loader is set on the spec object as an extra step.
> > Then every module that is used during startup would use
> > importlib.util.lazy_import() for importing instead of the normal import
> > statement.
>
> My experience is that:
> - you want lazy imports to be implicit, i.e. they should work using the
>   "import" statement and not any special syntax or function invocation
> - you want a blacklist and/or whitelist mechanism to restrict lazy
>   imports to a particular set of modules and packages, because some
>   modules may not play well with lazy importing (e.g. anything that
>   registers plugins, specializations -- think singledispatch --, etc.)
>
> For example, I may want to register the "tornado", "asyncio" and "numpy"
> namespaces / packages for lazy importing, but not the "numba" namespace
> as it uses registration mechanisms quite heavily.
>
> (and the stdlib could be part of the default lazy import whitelist)
>
> Regards
>
> Antoine.
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Speed mailing list
> Speed at python.org
> https://mail.python.org/mailman/listinfo/speed
>
>
> ------------------------------
>
> End of Speed Digest, Vol 36, Issue 1
> ************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/speed/attachments/20170625/dd4ad18a/attachment-0001.html>