Microthreads without Stackless?

[Patrick Maupin]

Michael Sparks wrote:

> The implied use case is this:
  <snip>

Ahh, I see -- code to be used in conjunction with a coroutine, rather
than code to be turned into a coroutine...

> Unless you think that no coroutine is ever going to use library code of some
> kind (either in-house or some shared library, or even just stuff in the
> standard python distro), then it is indicative of the sort of thing that
> could happily sit half way through a call stack in a stackful coroutine.

I agree that a naive implementation of coroutines could have exactly
the same sorts of issues with this that the threads module does, but
I'm not sure that it wouldn't be possible to do a better job.

> Note: I did say that in some circumstances they do make life simpler.
> However by adding them in at all, you _risk_ making reasoning about the
> entire system more complex, not simpler, depending on the implementation.
> It's *not* just sufficient to say that it's stackful.

The way you put this is why I thought you were arguing against adding
coroutines to the language.  I misread statements like "by adding them
in at all, you risk..." to mean that if they are added into the
language, all sorts of programs will catch them, like measles. 
Obviously, like threads, the fact that they are available doesn't
cause problems for packages that don't use them, and (I hope)
obviously, like threads, a good programmer can use them in a fashion
which won't endanger his whole program.


> That was the reason for the odd try/except/finally "hygiene". The reason for
> the bare except was to point out that it would catch any and all errors as
> you would normally with finally. You state that you think what I was doing
> wasn't good hygiene - however it's one of the few options you have if you
> use generators at present. I was aiming for consistency with that, not
> sanity :)

I missed the point that you were deliberately not using try/finally,
but in my defense, I thought the outer try/except should have been
try/except/else, with only the open() inside the try, and at a minimum
the inner try/except should have re-raised the exception after closing
the file.  These two things would make debugging process() _so_ much
easier :)

> I wouldn't say that's ever a reason for not putting something in a language.
> You'd never allow anyone to delete files on the disk otherwise... I was
> merely pointing out that you stated coroutines simplify reasoning, whereas
> in practice it can _break_ the _normal_ reasoning you have for the bulk of
> your code.

It's true they have to be used carefully, like threads.

> 
> >...  I submit that coroutines would be a satisfactory solution 
> 
> You seem to be arguing from the point of view that seems to think I'm
> against the idea of coroutines entering python, whereas all I was doing was
> pointing out that coroutines potentially break the reasoning for very
> common code, depending on how they are added. (if say generators were ever
> to be extended to be more stackful than they are)

Yes, my mis-reading of some of your arguments led me to believe that
you were saying that the tool was too dangerous to be made available,
rather than that programmers would need to be cautious with such a
tool.

> It's not, perhaps, quite as nice as "full" coroutines, but then it's
> nicer than the equivalent in many other languages.

Agreed!  And as you point out the current implementation of generators
is easy to reason about and make correct code "in the small".  
However, although I don't have any pressing need for coroutines
currently, if I _did_ I would certainly look at greenlets, or
Stackless, or any of a number of things before I  settled on any of
the (non-full) coroutine packages I've seen bandied about here,
because IMO these things would not scale in a very readable fashion.

Regards,
Pat