Generators and propagation of exceptions

[Raymond Hettinger]

On Apr 8, 8:55 am, r <nbs.pub... at gmail.com> wrote:
> I had a problem for which I've already found a "satisfactory"
> work-around, but I'd like to ask you if there is a better/nicer
> looking solution. Perhaps I'm missing something obvious.
>
> The code looks like this:
>
> stream-of-tokens = token-generator(stream-of-characters)
> stream-of-parsed-expressions = parser-generator(stream-of-tokens)
> stream-of-results = evaluator-generator(stream-of-parsed-expressions)
>
> each of the above functions consumes and implements a generator:
>
> def evaluator-generator(stream-of-tokens):
>   for token in stream-of-tokens:
>     try:
>        yield token.evaluate()           # evaluate() returns a Result
>     except Exception as exception:
>        yield ErrorResult(exception) # ErrorResult is a subclass of Result
>
> The problem is that, when I use the above mechanism, the errors
> propagate to the output embedded in the data streams. This means, I
> have to make them look like real data (in the example above I'm
> wrapping the exception with an ErrorExpression object) and raise them
> and intercept them again at each level until they finally trickle down
> to the output. It feels a bit like writing in C (checking error codes
> and propagating them to the caller).

You could just let the exception go up to an outermost control-loop
without handling it at all on a lower level.  That is what exceptions
for you: terminate all the loops, unwind the stacks, and propagate up
to some level where the exception is caught:

  while 1:
     try:
        results = evaluator-generator(stream-of-parsed-expressions)
        for result in results:
            print(result)
     except Exception as e:
        handle_the_exception(e)

OTOH, If you want to catch the exception at the lowest level and wrap
it up as data (the ErrorResult in your example), there is a way to
make it more convenient.  Give the ErrorResult object some identify
methods that correspond to the methods being called by upper levels.
This will let the object float through without you cluttering each
level with detect-and-reraise logic.

   class ErrorResult:
       def __iter__(self):
           # pass through an enclosing iterator
           yield self

Here's a simple demo of how the pass through works:

    >>> from itertools import *
    >>> list(chain([1,2,3], ErrorResult(), [4,5,6]))
    [1, 2, 3, <__main__.ErrorResult object at 0x2250f70>, 4, 5, 6]


> Any idea for a working and elegant solution?

Hope these ideas have helped.


Raymond