Config & ConfigParser

[Steven D'Aprano]

On Tue, 05 Mar 2013 18:15:20 -0600, Tim Chase wrote:

> On 2013-03-05 15:58, Chuck wrote:
>> Thanks Tim!  So much stuff I haven't thought of before.  Out of
>> curiosity, what's the benefit of caching the download,  instead of
>> downloading to the final destination?
> 
> If your connection gets interrupted, the server goes down, etc, you have
> a partial download.  If you've put it directly in the download path,
> your other programs see this partial download. However if your program
> can resume the download where it left off, once it's completed
> successfully, it can atomically move the file to your download location.
>  Thus your other programs only ever see all-or-nothing in the download
> directory.

That's not really a *cache* though.

Personally, I find programs that do that sort of cleverness annoying 
rather than helpful. More often than not, they are buggy and fail to 
clean up after themselves if the download is interrupted, so the secret 
download directory ends up filled with junk:

cat-video27~
cat-video27-1~
cat-video27-2~
cat-video27-3~

sort of thing.

Another problem with this tactic is that it makes it unnecessarily 
difficult to watch progress of the download, except via the application's 
official user interface. (If it gives you any interface for watching 
download progress, which is may not.) You have to locate the secret 
download directory, work out what file name the app is using for the 
temporary file (many apps obfuscate the file name), then watch that file 
grow until it suddenly disappears, at which point you then have to change 
directories to see if it reappeared where you actually wanted it to be, 
or was just deleted by something else.

A third problem: instead of having to worry about having enough disk 
space in one location, now you have to worry about having enough disk 
space in *two* locations.

I've even see a program download a large file into the temp location, 
*unsuccessfully* try to copy it into the final location, then delete the 
temp version.

Yet another problem: websites sometimes lie about the size of some files. 
So when you download them, the actual file ends (say) one byte short of 
what the webserver claims. There's nothing wrong with the file, it is 
actually complete, or at least recoverable (many formats, like JPEG, are 
remarkably resistant to damage), but your app will think it never 
completes and either never move it to the final destination, or worse, 
keep trying to download it over and over and over again.

Finally, moving from the temp directory to the final location is not 
necessarily an atomic operation. If it crosses file system boundaries, it 
is a two-step process. 

I think that the KISS principle applies here. If the user tells your app 
to download in some location, your app should download in that location.


-- 
Steven