[Import-SIG] Loading Resources From a Python Module/Package

[Donald Stufft]

> On Jan 31, 2015, at 1:29 PM, Barry Warsaw <barry at python.org> wrote:
> 
> On Jan 31, 2015, at 01:07 PM, Donald Stufft wrote:
> 
>> The reasons for not wanting to use a context manager are sort of intertwined
>> with each other.
>> 
>> The competitor to this function is something like:
>> 
>>   import os.path
>>   import time
>> 
>>   LOGO_PATH = os.path.join(os.path.dirname(__file__), "logo.gif")
>> 
>>   def print_logo_path():
>>       print(LOGO_PATH)
>> 
>> 
>>   while True:
>>       print_logo_path()
>>       time.sleep(1)
> 
> I'm just wondering if that's extracted from a real example or whether it's
> just a possible use case you'd want to support.  It's not a use case I've ever
> needed.
> 
> I reviewed a bunch of resource_filename() uses and in almost all cases it's
> 
> 1. Crafting a path-y thing for some other API that only takes paths.
> 2. Constructing a path for essentially shutil.copy()'ing the file somewhere
>   else (e.g. a test http server's file vending directory).
> 
> There are one or two where it might be inconvenient to use a context manager,
> but the majority of cases would be fine.

Yea, requests/certifi which doesn’t currently use resource_filename at all but
just constructs the path to the .pem file using __file__.

Also ensurepip and virtualenv (both the existing and the rewrite).

Almost every case where I had to access a resource file I end up needing to do
it multiple places and it was easier to just construct the path once and reuse
it.

> 
>> It makes it more akward to use anytime you need to use the file in multiple
>> locations or multiple times and since each context manager instance (in the
>> worst case) is going to need to get bytes, create a temp file, and write bytes
>> for each use of the context manager.
> 
> Perhaps it makes sense to either provide two APIs and/or implement a higher
> level API on top of a lower-level one?

i thought about doing it this way too, I didn’t just because I couldn’t really
imagine anyone really using the context manager when a simpler API was available
and I thought that having one way to do it was better. However I’m perfectly
happy to have two APIs if people think it’s important.

> 
>> The other thing is that for the "common" case, where the resource is available
>> on the file system already because we're just using a FileLoader, there is no
>> need for an atexit handler or a temporary file at all. The context manager
>> would only really exist for the uncommon case where we need to write the data
>> to a temporary file. Using the atexit handler allows us to provide the best
>> API for the common case, without too much problem for the uncommon case.
> 
> A context manager could also conditionalize the delete just like your proposal
> conditionalizes adding to the atexit handler.
> 
>> Yes it does mean that in certain cases the temporary files may be left behind,
>> particularly with kill -9 or segfaults or what have you. However that case
>> already exists, the only thing the context manager does is narrow the window
>> of case where a kill -9 or a segfault can leave temporary files behind.
> 
> Sure, but it reduces the window for leakage, which will probably be enough.
> 
> Cheers,
> -Barry

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA