[Python-Dev] this is what happens if you freeze all the modules required for startup

[M.-A. Lemburg]

On 18.04.2014 23:03, Ezio Melotti wrote:
> Hi,
> 
> On Thu, Apr 17, 2014 at 9:09 PM, Brett Cannon <bcannon at gmail.com> wrote:
>>
>>
>> On Thu Apr 17 2014 at 1:34:23 PM, Jurko Gospodnetić
>> <jurko.gospodnetic at pke.hr> wrote:
>>>
>>>    Hi.
>>>
>>> On 14.4.2014. 23:51, Brett Cannon wrote:
>>>> Now the question is whether the maintenance cost of having to rebuild
>>>> Python for a select number of stdlib modules is enough to warrant
>>>> putting in the effort to make this work.
>>>
>>>    I would really love to have better startup times in production, but I
>>> would also really hate to lose the ability to hack around in stdlib
>>> sources during development just to get better startup performance.
>>>
>>>    In general, what I really like about using Python for software
>>> development is the ability to open any stdlib file and easily go poking
>>> around using stuff like 'import pdb;pdb.set_trace()' or simple print
>>> statements. Researching mysterious behaviour is generally much much
>>> MUCH! easier (read: takes less hours/days/weeks) if it ends up leading
>>> into a stdlib Python module than if it takes you down into the bowels of
>>> some C module (think zipimport.c *grin*). Not to mention the effect that
>>> being able to quickly resolve a mystery by hacking on some Python
>>> internals leaves you feeling very satisfied, while having to entrench
>>> yourself in those internals for a long time just to find out you've made
>>> something foolish on your end leaves you feeling exhausted at best.
>>
>>
>> Freezing modules does not affect the ability to use gdb. And as long as you
>> set the appropriate __file__ values then tracebacks will contain even the
>> file line and location.
>>
> 
> Will the tracebacks only contain the line number or the line of code too?
> 
> I've seen tracebacks involving importlib,_bootstrap that didn't
> include the code, and I'm wondering if we will get something similar
> for all the other modules you are freezing:
> 
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 7, in <module>
>     import email
>   File "<frozen importlib._bootstrap>", line 1561, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1519, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 1473, in _find_module
>   File "<frozen importlib._bootstrap>", line 1308, in find_module
>   File "<frozen importlib._bootstrap>", line 1284, in _get_loader
>   File "<frozen importlib._bootstrap>", line 1273, in _path_importer_cache
>   File "<frozen importlib._bootstrap>", line 1254, in _path_hooks
> TypeError: 'NoneType' object is not iterable

Yes, this is what you get for frozen modules.

If you'd like to play around with a frozen stdlib this, you can
have a look at PyRun (http://pyrun.org), which does this for Python 2
and will hopefully work for Python 3.4 soonish too.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/