[Python-ideas] Disabling optimizations

[Ned Batchelder]

On 5/23/14 4:02 AM, Stephen Hansen wrote:
> On Thu, May 22, 2014 at 6:44 PM, Ned Batchelder <ned at nedbatchelder.com 
> <mailto:ned at nedbatchelder.com>> wrote:
>
>     I'm of the opinion that we don't need to segregate bytecode into
>     different files depending on the options used to create the
>     bytecode.  How often is the same program run in the same place
>     with different options at different times?  I'm happy to have
>     optimized and non-optimized code both written to .pyc files, and
>     if you are fiddling with the options like that, you should delete
>     your pyc files when you change the options.  If we come up with a
>     way to have the bytecode file-segregated, I'm OK with that too.
>
>
> What madness is this?
>
> Any suggestion that "you should delete your pyc files" strikes me as 
> remarkably wrongheaded. You shouldn't even have to think about pyc (or 
> pyo) files -- they're a convenience, not something there is any 
> expectation on anyone to *manage*. When I edit my .py file, I don't 
> have to go delete the pyc; I don't need to be sure to do a 'make 
> clean' like on some of my C projects. Python sees my source is 
> modified, and discards the compiled bit -- expecting anything more 
> from people using python is a serious thing.
>
> Things have gotten a bit more complex in modern Python with the 
> __pycache__ directory, yet still there is no expectation that users 
> *manage* these files. That's a bit shocking to me.
>
>     I definitely don't like the alternative that says unoptimized code
>     isn't written to disk at all.  If people want to solve the problem
>     that way, there is already a mechanism to avoid writing bytecode,
>     you can use it with the optimizer controls to achieve the effect
>     you want.
>
>
> I don't understand this point. It seems natural to me that if you have 
> an option to run code with optimizations disabled, its not written to 
> disk...: after all the entire assumption of the point is the code 
> isn't doing everything it can to be as efficient as it can. At that 
> point, what does speed matter? You've decided you want precise 
> traceable semantics even when its known that certain branches aren't 
> needed -- you want to trace the precise logic. Do you really then care 
> about the cost it takes to compile the source to bytecode?
>
> I get that there are reasons to not want optimizations, but I don't 
> get the desire to complicate the compilation and running step. 
> Optimizations on/off makes some sense: in testing environments and the 
> like. Its something else entirely to demand people manually delete 
> files, or where the burden is upon those who run the app/test 
> suites/etc to deal with files created as a side-effect of what they're 
> doing.
>

I may not have been clear, sorry: I would love to find a way to make 
this transparent to the user, and not have to have the user delete .pyc 
files. I was merely trying to make my requirements precise.  In my 
particular use case, having to delete .pyc files is not a problem.  If 
we can engineer it so that is not necessary, all the better.

The .pyc file already has a metadata that indicates the source timestamp 
and the version of the Python interpreter.  If those numbers don't mesh 
well with the Python source and interpreter that finds the pyc file, 
then the file is discarded transparently.  We could put the compilation 
options into the pyc file as well, and automatically discard the file if 
it had been made with different options than the running interpreter.

--Ned.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20140523/84c9ada8/attachment-0001.html>