[pypy-dev] pre-emptive micro-threads utilizing shared memory message passing?

[Kevin Ar18]

Sorry for not gettin back to you sooner.


I don't mind replying to the mailing list unless it annoys someone? Maybe some people could be interested by this discussion. 



You have a lot of questions! :) My answers are inline.


  


* Message passing
When you create a tasklet, you assign a set number of queues or streams to it (it can have many) and whether they extract data from them or write to them (they can only either extract or write to it as noted above). The tasklet's global namespace has access to these queues or streams and can extract or add data to them.

In my case, I look at message passing from the perspective of the tasklet. A tasklet can either be assigned a certain number of "in ports" and a certain number of "out ports." In this case the "in ports" are the .read() end of a queue or stream and the "out ports" are the .send() part of a queue or stream.




Sorry, I don't really understand what you're trying to explain here. Maybe an example could be helpful? :)


* Scheduler
For the scheduler, I would need to control when a tasklet runs. Currently, I am thinking that I would look at all the "in ports" that a tasklet has and make sure each one has some data. Only then would the tasklet be scheduled to run by the scheduler.




Couldn't all those ports (channels) be read one at a time, then the processing could be done? I don't exactly see the need to play with the scheduler. Channels are blocking. A tasklet will be anyway unscheduled when it tries to read on a channel in which no data is available.
 
http://www.jpaulmorrison.com/fbp/concepts.htm
Figure 3.6 and Figure 3.7 are a good example.
Let's say Figure 3.7 is the tasklet (the one with a local namespace and no access to global memory or memory in other tasklets).
IN, ACC, REJ are pointers to a shared memory location (from an implementation standpoint).
IN, ACC, REJ are either a queue or buffer/pipe/steam (from the perspective of the programmer).
The tasklet can only read/extract data from IN.
The tasklet can only write to ACC and REJ.
 
> Couldn't all those ports (channels) be read one at a time, then the processing could be done?
Not sure exactly, what you mean, but as shown in Figure 3.7, different parts of code will read or write to different ports at different times.
> A tasklet will be anyway unscheduled when it tries to read on a channel in which no data is available.
Good idea.  If there's no data to read, the tasklet can yield. ... but I need to know when the tasklet can be put back into the scheduler queue
 
Then again, I don't know how I will want to do the scheduler... and would like the low level primitives to explore different styles.
 
 
 
 
Anyways, at this point, I guess this whole discussion is not that important.  I should probably make something simpler for now just to try things out.  Then maybe I'll know if I want to even bother working on something better.   However, if you would like me to keep you up to date, I can contact you via email a few months from now.  (Let me know and I'll give you a different email to use). 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100812/a4abdb4d/attachment.html>