[Python-Dev] str object going in Py3K

[Walter Dörwald]

Bengt Richter wrote:

> On Wed, 15 Feb 2006 18:57:26 -0800, Guido van Rossum <guido at python.org> wrote:
> 
>> [...]
>> My expectation is that the Py3k standard I/O library will do all of
>> its own conversions on top of binary files anyway -- if you missed it,
>> I'd like to get rid of any ties to C's stdio.
>>
> Would the standard I/O module have low level utility stream-processing generators
> to do things like linesep normalization in text or splitlines etc? I.e., primitives
> that could be composed for unforseen usefulness, like unix pipeable stuff?
> 
> Maybe they could even be composable with '|' for unixy left->right piping, e.g., on windows
> 
>     for line in (os.open('somepath') | linechunker | decoder('latin-1')): ...
> 
> where os.open('path').__or__(linechunker) returns linechunker(os.open('path')),
> which in turn has an __or__ to do similarly. Just had this bf, but ISTM it reads ok.
> The equivalent nested generator expression with same assumed primitives would I guess be
> 
>     for line in decoder('latin-1')(linechunker(binaryfile('path'))): ...
> 
> which doesn't have the same natural left to right reading order to match processing order.

I'm currently implementing something like this, which might go into 
IPython. See http://styx.livinglogic.de/~walter/IPython/ipipe.py for 
code. (This requires the current IPython svn trunk)

Examples:

for f in ils("/usr/lib/python2.3/") | ifilter("name.endswith('.py')"):
    print f.name, f.size

for p in ipwd | ifilter("shell=='/bin/false'") | isort("uid") | \
    ieval('"%s (%s)" % (_.name, _.gecos)'):
    print p

The other part of the project is a curses based browser for the output 
of these pipelines. See 
http://styx.livinglogic.de/~walter/IPython/newdir.gif for a screenshot 
of the result of ils("/usr/lib/python2.3/") | 
ifilter("name.endswith('.py')")

Bye,
    Walter Dörwald