[Python-Dev] PEP 488: elimination of PYO files

[Nick Coghlan]

For the record here: +1 on the PEP from me (the comments I made on
import-sig have already incorporated into this version of the PEP)

On 8 March 2015 at 08:03, Brett Cannon <brett at python.org> wrote:
>
>
> On Sat, Mar 7, 2015 at 12:39 PM Scott Dial <scott+python-dev at scottdial.com>
> wrote:
>>
>> On 2015-03-06 11:34 AM, Brett Cannon wrote:
>> > This PEP proposes eliminating the concept of PYO files from Python.
>> > To continue the support of the separation of bytecode files based on
>> > their optimization level, this PEP proposes extending the PYC file
>> > name to include the optimization level in bytecode repository
>> > directory (i.e., the ``__pycache__`` directory).
>>
>> As a packager, this PEP is a bit silent on it's expectations about what
>> will happen with (for instance) Debian and Fedora packages for Python.
>> My familiarity is with Fedora, and on that platform, we ship .pyc and
>> .pyo files (using -O for the .pyo). Is it your expectation that such
>> platforms will still distribute -O only? Or also -OO? In my world, all
>> of the __pycache__ directories are owned by root.
>
>
> I assume they will generate all .pyc files at all levels, but I don't if
> it's my place to dictate such a thing since bytecode files are an
> optimization to Python itself and do not influence how people interact with
> the interpreter like with PEP 394 (The "python" Command on Unix-Like
> Systems).

= Fedora near term =

Nothing significant will change for RPMs with policy compliant
(https://fedoraproject.org/wiki/Packaging:Python#Byte_compiling) spec
files, even after switching to Python 3.5. The only different will be
that the current .pyo files will be replaced with files using the new
PEP 488 naming scheme. Folks using custom spec files with their own
pattern matches that assume "pyo" may need to adjust them (e.g. to
grab everything in __pycache__, as recommended in the packaging
guidelines)

= Fedora long term =

After the switch to Python 3.5 (which, given the release schedule, I
currently expect will be in Fedora 24 early next year), the RPM Python
build macros *might* be changed to generate -OO files in addition to
-O ones, making it easier to run Python applications in -OO mode to
reduce memory usage. The PEP doesn't mandate such a change, but it
does enable it. This would have the downside of making every Python
package in the distro slightly larger (all 15k+ of them), so there'd
need to be a clear and compelling pay-off to justify that cost.

While folks may be tempted to say "disk space is cheap", VM and
container provisioning latency are performance critical in a number of
use cases, so making the Fedora Cloud and Atomic Host images even
slightly larger isn't likely to win us any friends, so actually making
such a change would require compelling benchmarks demonstrating
reductions in runtime memory usage that justify the larger image size,
as well as confirming that the change doesn't place undue pressure on
the DVD ISO image contents.

This isn't really the right list for more in-depth discussion of what
Fedora *might* do though - any discussion will be entirely theoretical
until after Python 3.5 comes out, and even then the appropriate list
would be python-devel at lists.fedoraproject.org rather than here.

Regards,
Nick.

P.S. For those that aren't already aware, I became a voting member of
Fedora's Environments & Stacks Working Group several months ago and
now work closely with Slavek Kabrda (the Fedora Python maintainer,
amongst other things) on Python's integration into Fedora and its
derivatives.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia