[Python-Dev] __pycache__ creation

[Guido van Rossum]

On Mon, Mar 22, 2010 at 12:20 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 22, 2010, at 02:02 PM, Ron Adam wrote:
>
>>If I understand correctly, we would have the current mode as the default, and
>>can trigger __pycache__ behavior simply by manually creating a __pycache__
>>directory and deleting any byte-code files in the module/program directory.

Huh? Last time I looked weren't we going to make __pycache__ the
default (and eventually only) behavior?

>>I like this, it is easy to understand and can be used without messing with
>>flags or environment variables.
>
> Well, for a package with subpackages, it gets more complicated.  Definitely
> not something you're likely to do manually.  Antoine's suggestion of 'python
> -m compileall --pycache' would work, but I think it's also obscure enough that
> most Python users won't get the benefit.

I see only two reasonable solutions for __pycache__ creation -- either
we change all setup/install scripts (both for core Python and for 3rd
party packages) to always create a __pycache__ subdirectory for every
directory (including package directories) installed; or we somehow
create it the first time it's needed.

But creating it as needed runs into at least similar problems with
ownership as creating .pyc files when first needed (if the parent
directory is root-owned a mere mortal can't create it at all). So even
apart from the security issue (which I haven't thought about deeply) I
think precreation should at least be an easily accessible option both
for the core (where it can be done by compileall) and for 3rd party
packages (where I guess it's up to distutils or whatever install
mechanism is used).

-- 
--Guido van Rossum (python.org/~guido)