How to implement a pipeline...??? Please help

Wed Aug 25 01:21:34 EDT 2010

On 11 Ago, 01:01, Ritchy lelis <ritchy_g... at hotmail.com> wrote:
> On 7 Ago, 07:30, Dennis Lee Bieber <wlfr... at ix.netcom.com> wrote:
>
>
>
>
>
> > On Fri, 6 Aug 2010 16:47:58 -0700 (PDT), Ritchy lelis
> > <ritchy_g... at hotmail.com> declaimed the following in
> > gmane.comp.python.general:
>
> > > Guys i'm asking if it's possible for a generic function for a
> > >pipeline, all the suggestions ideas are welcome.
>
> >         I've not done metaclasses, so don't know if a metaclass would be of
> > use... Since it appears one would have to have something that generates
> > functions on the fly based upon inputs...
>
> >         I look at details on what you are trying to create and see a
> > series-parallel circuit architecture with clocked stages and delay lines
> > for the summing of the outputs. That is, when the clock ticks, every
> > stage in the unit does its processing and provides and output... On the
> > next tick, those outputs become inputs to the subsequent stage and the
> > process repeats
>
> > > Also if you have a specific solution for my problem I will be grateful
> > > for it here.
>
> >         I'm still blinking at the thought of doing analog to digital
> > conversion in Python! I sure wouldn't hold out much hope of real-time
> > signal processing. There is a reason ADC and DAC are done in hardware,
> > even if the intermediate processing is with general purpose processors.
>
> >         I'm also not sure I understand the
>
> > np.linspace(1,1, Inc)...
>
> >         Looking up documentation implies you are trying to create a vector
> > of "inc" length, evenly populated by values between 1 and 1... which
> > means a vector of all 1s... Might it not be faster to just
>
> >         v = [1] * inc
>
> > and pass v to some numpy method for conversion from Python list to numpy
> > vector?
>
> > (the -1 to 1 at least makes sense)
>
> >         Me? I'd probably create a class in which the __init__() takes a
> > value specifying the number of stages. It would than create suitable
> > lists to track values, some sort of counter (c), a Semaphore (s)
> > [initialized at 0 -- ie, already acquired/block],  a thread for EACH
> > stage, and an Event (e) object
>
> >         Each stage thread, initialized with its position in thepipeline.
> > The threads perform an e.wait() call. They also, after the wait is
> > released, perform an e.clear() call. As their processing, they each grab
> > from thepipeline"input" list the current value for their position.
> > After processing they update their position in the "output" list(s),
> > they decrement the counter c (maybe put a lock around access to c). The
> > thread that decrements c to 0 is responsible for "releasing" the
> > semaphore.
>
> >         The main code of the class instance is responsible for a loop that
> > does: set up the "input" list based on current value of stage outputs,
> > initialize c to the count of stages (minus 1?), e.set() to signal all
> > stage threads to process the current conditions, s.acquire() to block
> > until the last processed thread (by c hitting 0) does s.release(). It
> > then collects the output lists, does whatever shifting is needed to
> > prepare for the next cycle...
>
> >         Actually, that "loop" may not be a loop so much as a method off the
> > class like
>
> >         digitalOutput = adcInstance.step(analogInput)
>
> > which is, itself, in a loop.
>
> > That is, something like...
>
> > myADC = ADC(stages=10)
> > while True:
> >         voltage = getNextAnalogInput()
> >         digital = myADC.step(voltage)
> >         outputDigitalValue()
>
> >         Obviously I've not taken the time to actually lay out all the
> > instance lists needed for inputs and outputs, nor the code of threads
> > (while one can create the first "stages-1" threads with a loop, the
> > final stage needs a discrete creation)
>
> >         When one finds that the threading solution is really slow (though
> > understandable in terms of the hardware circuit -- one thread per stage
> > makes the stages easy to code), THEN one might try to figure out how to
> > implement an iterative version... I suspect using numpy would be the
> > third optimization -- removing iteration by using parallel vector
> > operations.
> > --
> >         Wulfraed                 Dennis Lee Bieber         AF6VN
> >         wlfr... at ix.netcom.com    HTTP://wlfraed.home.netcom.com/
>
> Hi
>
> First of all i would like to thank you for your time and help.
>
> I appreciate your suggestions and they seems very good to me... The
> only problem it's that i'm newbie at python programming, but still
> interested to learn more and more...
>
> I already got a solution through harsh but it worked. at least I got
> the wave form i wanted.
>
> Next i'm going to let you'll my solution here. It's rudimentary but it
> work's.
>
> Seems very perceptible that's why i will not explain it right now but
> if in case of doubt I will:
>
> PipelineFunction:
>
> from flash1b5 import flash1b5
> from flash1b5 import *
> from flash2b import flash2b
> from flash2b import *
> import matplotlib.pyplot as plt
> import numpy as np
> from pylab import *
>
> if __name__ == "__main__":
>
>     Inc = raw_input("Valor do Incrimento = ")
>
>     Vref = np.linspace(1,1, Inc)
>     Vi = np.linspace(-1,1, Inc)
> #    x = np.linspace(-1,1, Inc)
> #    Vi = 1*sin(2*pi*(x-1/4))
>     Cs = np.linspace(3e-12, 3e-12, Inc)
>     Cf = np.linspace(3e-12, 3e-12, Inc)
>
>     f1 = flash1b5(Vi, Vref, Cs, Cf)
>
>     Vin1 = f1[0]
>     Vt1 = f1[1]
>     Vd1 = f1[2]*256
>
>     f2 = flash1b5(Vt1, Vref, Cs, Cf)
>
>     Vin2 = f2[0]
>     Vt2 = f2[1]
>     Vd2 = f2[2]*128
>
>     f3 = flash1b5(Vt2, Vref, Cs, Cf)
>
>     Vin3 = f3[0]
>     Vt3 = f3[1]
>     Vd3 = f3[2]*64
>
>     f4 = flash1b5(Vt3, Vref, Cs, Cf)
>
>     Vin4 = f4[0]
>     Vt4 = f4[1]
>     Vd4 = f4[2]*32
>
>     f5 = flash1b5(Vt4, Vref, Cs, Cf)
>
>     Vin5 = f5[0]
>     Vt5 = f5[1]
>     Vd5 = f5[2]*16
>
>     f6 = flash1b5(Vt5, Vref, Cs, Cf)
>
>     Vin6 = f6[0]
>     Vt6 = f6[1]
>     Vd6 = f6[2]*8
>
>     f7 = flash1b5(Vt6, Vref, Cs, Cf)
>
>     Vin7 = f7[0]
>     Vt7 = f7[1]
>     Vd7 = f7[2]*4
>
>     f8 = flash1b5(Vt7, Vref, Cs, Cf)
>
>     Vin8 = f8[0]
>     Vt8 = f8[1]
>     Vd8 = f8[2]*2
>
>     f2b = flash2b(Vt8, Vref)
>
>     Vin2b = f2b[0]
>     Vd2b = f2b[1]*1
>
>     Vd = Vd1+Vd2+Vd3+Vd4+Vd5+Vd6+Vd7+Vd8+Vd2b
>
> ##    print 'Vin = ',Vin
> ##    print 'Vt = ',Vt
> ##    print 'Vd = ',Vd
> ##
> #    fig1 = figure(1,figsize=(8,5))
> #    ax1 = fig1.add_subplot(211, autoscale_on=False, xlim=(-1,1),
> ylim=(-1,1))
> #    ax1.plot(Vin1, Vt4, lw=2, color='blue')
> #    grid (True); title('FLASH 1.5 BIT',fontsize =
> 16);ylabel('Vout_Residuo')
> ##
> ##    ax1.annotate('00', xy=(-0.5, 0.5))
> ##    ax1.annotate('01', xy=(0.0, 0.5))
> ##    ax1.annotate('11', xy=(0.5, 0.5))
> #
> ##    hold(True)
> #
>     fig2 = figure(1,figsize=(8,5))
>     ax2 = fig2.add_subplot(111, autoscale_on=True)
>     ax2.plot(Vin1, Vd, lw=2, color='red')
>     grid (True); xlabel('Vin');ylabel('Vout_Digital')
> #
> ##    ax2.annotate('00 --> 0', xy=(-0.5, 0.1))
> ##    ax2.annotate('01 --> 1', xy=(0.0, 1.1))
> ##    ax2.annotate('11 --> 2', xy=(0.5, 2.1))
> ##
> #
>     plt.show()
> --------------------------------------------------------------------------
>
> About the answer i got for my last post:
>
>  1 -
>     v = [1] * inc
>
>
>
> > and pass v to some numpy method for conversion from Python list to numpy
> > vector?
>
> Yes i agree with you, but at the time i made it i found that function
> (Linspace) that could do what i was loking for and i didn't worry
> about search for a better solution. but even if wanted to, i'm new at
> the programming language and i don't know how to convert a list to a
> vector in numpy. but still open for tips/tricks that could help me.
>
> 2 -
>
> >         Me? I'd probably create a class in which the __init__() takes a
> > value specifying the number of stages. It would than create suitable
> > lists to track values, some sort of counter (c), a Semaphore (s)
> > [initialized at 0 -- ie, already acquired/block],  a thread for EACH
> > stage, and an Event (e) object
> >         Each stage thread, initialized with its position in thepipeline.
> > The threads perform an e.wait() call. They also, after the wait is
> > released, perform an e.clear() call. As their processing, they each grab
> > from thepipeline"input" list the current value for their position.
> > After processing they update their position in the "output" list(s),
> > they decrement the counter c (maybe put a lock around access to c). The
> > thread that decrements c to 0 is responsible for "releasing" the
> > semaphore
>
> I did understand your algorithm/idea and that's what i want to
> implement here (you couldn't be more right). but i don't now how to
> implement the funcs e.wait() call and e.clear() call. they already
> exists? i have to create them?
>
> if I may, i would like to ask more help on this one please. I hope not
> to be bothering you :(

hi friend Dennis Lee Bieber

I have watching your code sujestion and now i can understand more of
what you have there..

I have made some chances into it, for my better understanding of your
point of view and adapted for it to fits my needs...

i would like to show you them but i need to now if you still
interested in the challenge it self and also in helping me with your
extraordinary tips.

Can i count on your help? I hope so gratefully.

I'll be waiting for a message so we can proceed.

Cheers.