Can python read up to where a certain pattern is matched?

[F. Petitjean]

On 07 Mar 2004 22:35:55 GMT, F. Petitjean <littlejohn.75 at news.noos.fr> wrote:
>> 
>> Please give a hint.  Thank you!
>> 
> the hint :
> import itertools
> help(itertool.takewhile)
> 
> # not tested (no python 2.3 on Debian gateway at home)
> 
> import itertools
< snip a piece of code full of bugs >
> 
> text = """\
> Some words here, and some other words. Then another
> segment follows, and more. This is a question, a junk
> question, followed by a question mark?"""
> 
> for sentence in readsentence(text):
>     print sentence

A lightly tested version :

def readsentence(iterable, ends = (".", "!", "?"), yield_fn=''.join):
    """generator function which yields sentences terminated by ends"""
    end_pred = ends
    if not callable(ends):
        end_pred = lambda c : c in ends
    it = iter(iterable)
    sentence = []
    add = sentence.append
    while True:
        c = it.next()
        if not end_pred(c):
            add(c)
        else:
            # Now add the item terminating the sentence
            add(c)
            t = tuple(sentence)
            if callable(yield_fn):
                t = yield_fn(t)
            yield t
            sentence[:] = []
            # add = sentence.append

    if sentence:
        t = tuple(sentence)
        if callable(yield_fn):
            t = yield_fn(t)
        yield t


Regards