[py-svn] r10419 - py/branch/py-collect/documentation

[hpk at codespeak.net]

Author: hpk
Date: Fri Apr  8 03:24:18 2005
New Revision: 10419

Modified:
   py/branch/py-collect/documentation/future.txt
Log:
refactor and extend the future chapter 
with some real fun stuff 



Modified: py/branch/py-collect/documentation/future.txt
==============================================================================
--- py/branch/py-collect/documentation/future.txt	(original)
+++ py/branch/py-collect/documentation/future.txt	Fri Apr  8 03:24:18 2005
@@ -10,7 +10,8 @@
 statements within this document - even if they sound factual -
 mostly just express thoughts and ideas. They not always refer to 
 real code so read with some caution.  This is not a reference guide
-(tm).* 
+(tm). Moreover, the order in which appear here in the file does 
+not reflect the order in which they may be implemented.* 
 
 .. _`general-path`: 
 .. _`a more general view on path objects`:
@@ -355,29 +356,29 @@
 Refactor path implementations to use a Filesystem Abstraction 
 ============================================================= 
 
-It seems a good idea to refactor all python implementations to
-use an internal Filesystem abstraction.  There would be
-Filesystem implementations for e.g. local, subversion 
-and subversion working copy filesystems. Today the
-according code is scattered through the actual path-handling 
-code. 
-
-On a related note, Armin has hacked `pylufs`_ which allows to 
-write linux filesystems at pure python level.  The idea is that
-the new filesystems objects would be implemented in a way that 
-make them directly usable for the linux-filesystem glue code. 
-Implementing a `memoryfs`_ or a `dictfs`_ should then give
-you for free: a filesystem mountable at linux kernel level
-as well as a uniform "path" object allowing you to traverse
-and manipulate the filesystem tree.  At some point it might
-become interesting to interface to some `reiserfs v4 stuff`_
-at the Filesystem level.  
-
+It seems like a good idea to refactor all python implementations to
+use an internal Filesystem abstraction.  The current code base
+would be transformed to have Filesystem implementations for e.g. 
+local, subversion and subversion "working copy" filesystems. Today 
+the according code is scattered through path-handling code. 
+
+On a related note, Armin Rigo has hacked `pylufs`_ which allows to 
+implement kernel-level linux filesystems with pure python. Now 
+the idea is that the mentioned filesystem implementations would 
+be directly usable for such linux-filesystem glue code. 
+
+In other words, implementing a `memoryfs`_ or a `dictfs`_ would 
+give you two things for free: a filesystem mountable at kernel level
+as well as a uniform "path" object allowing you to access your
+filesystem in convenient ways.  (At some point it might
+even become interesting to think about interfacing to  
+`reiserfs v4 features`_ at the Filesystem level but that
+is a can of subsequent worms).  
 
 .. _`memoryfs`: http://codespeak.net/svn/user/arigo/hack/pylufs/memoryfs.py
 .. _`dictfs`: http://codespeak.net/pipermail/py-dev/2005-January/000191.html 
 .. _`pylufs`: http://codespeak.net/svn/user/arigo/hack/pylufs/
-.. _`reiserfs v4 stuff`: http://www.namesys.com/v4/v4.html
+.. _`reiserfs v4 features`: http://www.namesys.com/v4/v4.html
 
 
 Improve and unify Path API 
@@ -390,23 +391,49 @@
 which will limit traversal to subdirectories. Example:: 
 
     x = py.path.local.get_tmproot()
-    for x in x.visit('bin', maxdepth=N): 
+    for x in p.visit('bin', stop=N): 
         ... 
 
-This will with find all files or directories  named 'bin', 
-depending on the values of ``maxdepth``:: 
+This will yield all file or directory paths whose basename
+is 'bin', depending on the values of ``stop``:: 
+
+    p                       # stop == 0 or higher (and p.basename == 'bin')
+    p / bin                 # stop == 1 or higher
+    p / ... / bin           # stop == 2 or higher
+    p / ... / ... / bin     # stop == 3 or higher
+
+The default for stop would be `255`. 
 
-    x                       # maxdepth == 0 (and x.basename == 'bin')
-    x / bin                 # maxdepth == 1 
-    x / ... / bin           # maxdepth == 2
-    x / ... / ... / bin     # maxdepth == 3
+But what if `stop < 0`?  We could let that mean to go upwards:: 
 
-What if `x < 0`?  We could let that indicate just going upwards. 
-    for x in x.visit('py/bin', maxdepth=-255): 
+    for x in x.visit('py/bin', stop=-255): 
         # will yield all parent direcotires which have a 
         # py/bin subpath 
-        #
 
-Is `maxdepth` a good name? Should this functionality live
-on a different method alltogether? 
+visit() returning a lazy list? 
+------------------------------ 
+
+There is a very nice "no-API" `lazy list`_ implementation from 
+Armin Rigo which presents a complete list interface, given some 
+iterable.  The iterable is consumed only on demand and retains 
+memory efficiency as much as possible.  The lazy list 
+provides a number of advantages in addition to the fact that
+a list interface is nicer to deal with than an iterator. 
+For example it lets you do:: 
+
+    for x in p1.visit('*.cfg') + p2.visit('*.cfg'): 
+        # will iterate through all results 
+
+Here the for-iter expression will retain all lazyness (with
+the result of adding lazy lists being another another lazy
+list) by internally concatenating the underlying
+lazylists/iterators.  Moreover, the lazylist implementation
+will know that there are no references left to the lazy list
+and throw away iterated elements.  This makes the iteration
+over the sum of the two visit()s as efficient as if we had 
+used iterables to begin with! 
+
+For this, we would like to move the lazy list into the 
+py lib's namespace, most probably at `py.builtin.lazylist`. 
 
+.. _`lazy list`: http://codespeak.net/svn/user/arigo/hack/collect