Generators and propagation of exceptions

[Ian Kelly]

On Fri, Apr 8, 2011 at 9:55 AM, r <nbs.public at gmail.com> wrote:
> I had a problem for which I've already found a "satisfactory"
> work-around, but I'd like to ask you if there is a better/nicer
> looking solution. Perhaps I'm missing something obvious.
>
> The code looks like this:
>
> stream-of-tokens = token-generator(stream-of-characters)
> stream-of-parsed-expressions = parser-generator(stream-of-tokens)
> stream-of-results = evaluator-generator(stream-of-parsed-expressions)
>
> each of the above functions consumes and implements a generator:
>
> def evaluator-generator(stream-of-tokens):
>  for token in stream-of-tokens:
>    try:
>       yield token.evaluate()           # evaluate() returns a Result
>    except Exception as exception:
>       yield ErrorResult(exception) # ErrorResult is a subclass of Result
>
> The problem is that, when I use the above mechanism, the errors
> propagate to the output embedded in the data streams. This means, I
> have to make them look like real data (in the example above I'm
> wrapping the exception with an ErrorExpression object) and raise them
> and intercept them again at each level until they finally trickle down
> to the output. It feels a bit like writing in C (checking error codes
> and propagating them to the caller).
>
> OTOH, if I don't intercept the exception inside the loop, it will
> break the for loop and close the generator. So the error no longer
> affects a single token/expression but it kills the whole session. I
> guess that's because the direction flow of control is sort of
> orthogonal to the direction of flow of data.
>
> Any idea for a working and elegant solution?


I guess that depends on what you would want to do with the exceptions
instead.  Collect them out-of-band?  Revising your pseudo-code:

errors = []

stream-of-tokens = token-generator(stream-of-characters, errors)
stream-of-parsed-expressions = parser-generator(stream-of-tokens, errors)
stream-of-results = evaluator-generator(stream-of-parsed-expressions, errors)

def evaluator-generator(stream-of-tokens, errors):
 for token in stream-of-tokens:
   try:
      yield token.evaluate()           # evaluate() returns a Result
   except Exception as exception:
      errors.append(exception)
      # or:
      # errors.append(EvaluatorExceptionContext(exception, ...))


Cheers,
Ian