Agnostic fetching

[Ivan Ven Osdel]

>----- Original Message -----
>From: "Diez B. Roggisch" <deets at nospam.web.de>
>To: python-list at python.org
>Sent: Saturday, August 2, 2008 11:05:07 AM GMT -06:00 US/Canada Central
>Subject: Re: Agnostic fetching

>Bruce Frederiksen schrieb:
> On Fri, 01 Aug 2008 17:05:00 -0700, jorpheus wrote:
> 
>> OK, that sounds stupid. Anyway, I've been learning Python for some
>> time now, and am currently having fun with the urllib and urllib2
>> modules, but have run into a problem(?) - is there any way to fetch
>> (urllib.retrieve) files from a server without knowing the filenames?
>> For instance, there is smth like folder/spam.egg, folder/
>> unpredictable.egg and so on. If not, perhaps some kind of glob to
>> create a list of existing files? I'd really appreciate some help,
>> since I'm really out of my (newb) depth here.
> 
> You might try the os.path module and/or the glob module in the standard
> python library.

>Not on remote locations. The only work on your local filesystem.

>Diez

Here's a function I wrote for checking remote or local file existence. It works for me but admittedly I haven't tested many cases with it. Also its currently specific to an http URI scheme.

def fileExists(self, fileUrlPath):
    fileExists = False
    if "http:" in fileUrlPath.lower():
        #We don't want to open the file so ask the header if the
        #file exists
        urlParts = urlparse(fileUrlPath)
        host = urlParts[1]
        http = httplib.HTTP(host)
        http.putrequest("HEAD", fileUrlPath)
        http.putheader("Host", host)
        http.endheaders()
        errorcode, errormessage, headers = http.getreply()
        if errorcode == 200:
            fileExists = True
    else:
        fileExists = path.exists(fileUrlPath)

    return fileExists

-- 
Ivan Ven Osdel
http://datasyncsuite.com