[AstroPy] Temporary files leaked by long-running process that calls astropy.utils.data.download_file

[Erik Bray]

On 09/28/2015 10:17 PM, Singer, Leo P. (GSFC-661.0)[OAK RIDGE ASSOCIATED
UNIVERSITIES (ORAU)] wrote:
> Hi,
>
> I have a process that is meant to run for a few days to months at a time,
> that is leaking temporary files whenever I download files using the functions
> in astropy.utils.data. I found no leaks in my own code. Then I looked in
> astropy.utils.data.get_readable_fileobj, and I found that the temporary files
> that it creates always make their way into the delete_fds list, and could not
> be leaked. Then I finally thought to look in astropy.utils.data.download_file
> itself, and found that it is leaking the files—at least until the interpreter
> exits cleanly!
>
> if conf.delete_temporary_downloads_at_exit: global _tempfilestodel
> _tempfilestodel.append(local_path)
>
> I find the astropy.utils.data module very convenient, and I would like to
> continue to use it in this long-running process. Would the maintainers be
> open to a patch that adds an option to delete these temporary files
> immediately, instead of at cleanup?

Hi Leo,

Thanks for investigating this.  For what it's worth, discussion about 
development of the astropy package itself is usually better directed toward the 
astropy-dev mailing list:

https://groups.google.com/forum/#!forum/astropy-dev

Or directly to the issue tracker.

But yes, why don't you open an issue for that.  Depending on the context it may 
not always be easy to automatically delete files when they're "done" being used, 
and it would be up to the user to delete them.  But there are other cases where 
this would make sense.

Thanks,
Erik