fiber(cooperative multi-threading)

[Michael Sparks]

Hi,


> It just works, but using native Python threads for non-preemptive
> threading is not cost-effective. Python has generator instead but it
> seemed to be very restricted for general scripting. I wish I could
> write nested (generator) functions easily at least.
> 
> Is there any plan of implementing real (lightweight) fiber in Python?

Please take a look at Kamaelia. Generators work extremely well, and are
extremely useful to work with:

   * http://kamaelia.sourceforge.net/Home

(I need to update/improve the website...)

If you *must* have nested generators (it's rarely useful) you can do that,
but just need to have your "thing running the generators" understand
a "don't run me next, run this next" message. Our message is called
WaitComplete, and you can see it used here (in a non-trivial useful
system):

https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/branches/private_MPS_Scratch/Apps/Kamaelia-Grey/App/greylisting.py

(I could point at trivial examples, but prefer something real)

The shortest overall description is here:
   * http://kamaelia.sourceforge.net/t/TN-LightTechnicalIntroToKamaelia.pdf

Cookbook examples:
   * http://kamaelia.sourceforge.net/Cookbook

List of dozens of reusable (all with each other) components:
   * http://kamaelia.sourceforge.net/Components

How to build your own (non-optimised) core:
   * http://kamaelia.sourceforge.net/MiniAxon/

Ruby (mini) version:  (for comparison :-)
https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/trunk/Code/Ruby/miniaxon.rb

Experimental process based (as well as thread & generator based) version:

http://yeoldeclue.com/cgi-bin/blog/blog.cgi?rm=viewpost&nodeid=1196129474

(Going to move to process based components for static segmentation of an
application across process boundaries to give explicit multicore support.
May move to automatic distribution at some point, but static is an easy win
as the above example shows :-)

It's not as lightweight as stackless python's microthreads, but it's pretty
close. The scaling style is pretty similar as well - cf:

http://www.rhonabwy.com/wp/2007/11/13/generator-based-concurrent-programming-in-python/
I also blogged about that here:
http://yeoldeclue.com/cgi-bin/blog/blog.cgi?rm=viewpost&nodeid=1195688924

(Someone curious about comparing Kamaelia to stackless. Interesting to see
the same scaling curve, but unsurprisingly stackless wins :-) Kamaelia
however works with standard python from version 2.2.late onwards :-) )

Interestingly recently discovered that Kamaelia's been slowly reinventing a
system the UK's MOD have been using for around 30 years to make concurrent
systems easier to build.... (mascot) :-)

Merry Christmas :-)


Michael
--
http://yeoldeclue.com/blog
http://kamaelia.sourceforge.net/Developers/