Virtual Filesystem (was: Adding ftputil to the Python distribution?)

[Bengt Richter]

On 8 Nov 2002 01:20:16 -0800, stephan.diehl at gmx.net (Stephan Diehl) wrote:

>Jacob Smullyan <smulloni at bracknell.smullyan.org> wrote in message news:<slrnasle8i.v6v.smulloni at bracknell.smullyan.org>...
>> In article <f199e60.0211070207.5ed58c25 at posting.google.com>, Stephan Diehl wrote:
>> > Talking about a future VFS: is there already somethings planned? It
>> > would be very nice to access a local filesystem, remote filesystem
>> > (over ssh/scp), ftp server, webdav server,...  with the same set of
>> > commands.
>> 
>> SkunkWeb has a vfs package (which can be used independently) that you
>> might be interested in:
>> 
>> http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/skunkweb/skunkweb/pylibs/vfs/
>> 
>> Features:
>> 
>> 1. implementations for local fs, ftp, zip/tar/tgz archives, and a
>> "multifs" in which it is possible to mount other fses with
>> considerable flexibility (for instance, it is possible for a mount to
>> change its mount point dynamically, which is useful in the context of
>> web server document roots, where you may want a document root to shift
>> for a particular request while other mounts remain in the same place
>> relative to that shift). 
>> 
>> 2. Writing a new fs is fairly trivial.  The interface is adequate for
>> many needs, and supports read-only as well as read-write fses. 
>> 
>> 3. an import hook (using Gordon MacMillan's superb iu.py) which
>> enables you to import python modules from any fs implementation. This
>> is actually a big gain, from my point of view, as struggling with
>> import hooks is something I'd rather do once and then forget about.
>> Apparently Python Labs feels the same way, which is why we're still
>> stuck with having to write them! 
>> 
>> There is some code, namely, PathPropertyStore and its illegitimate
>> offspring, that you might as well ignore.  I was fiddling at one time
>> with the notion of providing a general way of storing properties
>> associated with vfs files, but came to the conclusion that this level
>> of abstraction is not such a hot place, data-integrity-wise, to
>> implement that.
>> 
>> Cheers,
>> 
>> Jacob Smullyan
>
>Thanks, that's excactly what I was thinking of/looking for. And it
>shows to me that there is nearly nothing that has not been written in
>one form or another. I'd really like to see a vfs in the standard
>python distribution.
I too. But I believe any given implementation will implicitly reflect
conventions with respect to operations in abstract name space, and I think
that ought to be laid out in a PEP before adopting any particular implementation.

Otherwise there is likely to be special limitations, e.g., not being able
to delegate parsing of path tails to other vfs's mounted within the first, etc.
Also a virtual name space brings with it opportunity for orderly conventions
of use as opposed to ad hoc quilts of usage. A PEP can at least draw attention
to LSB/FHS type issues.

>
>Maybe, one day, we'll have a central register for python libraries. It
>seems to me that to many people (including me) are reinventing the
>wheel all the time because they just can't find the right library for
>the task at hand and just don't know project xyz that already solved
>the problem (and doesn't show up when googling)
>
As a habitual wheel re-inventor, I'll second the thought ;-)

OTOH, I think re-inventing has positive aspects sometimes. A certain
amount of reinventing is necessary for natural selection to have something
to select from in the evolutionary improvement of things. OTOH2, for
well-explored problems, it's an indulgence to reinvent something if you can find
and use something already implemented. (Of course, it _is_ satifying to find
that you have re-invented, even approximately, an algorithm that some real guru
has previously invented, and feel that you walked in the same ancient hallowed land,
even as you realize that you could have spent your time better for practical purposes).

The vfs above is very close to what I was talking about recently. But not
exactly, and not quite from the same POV. And any discussion of general
or genericized (v)file access will garner reminders that lisp has a lot
of prior art ;-)

My main thought before was that a virtual name space can be platform
independent (or at least contain such a subspace), and that could serve
the purpose of platform independence for Python.

Regards,
Bengt Richter