threading - race condition?

[skunkwerk]

On May 9, 12:12 am, John Nagle <na... at animats.com> wrote:
> skunkwerk wrote:
> > i'm getting the wrong output for the 'title' attributes for this
> > data.  the queue holds a data structure (item name, position, and list
> > to store results in).  each thread takes in an item name and queries a
> > database for various attributes.  from the debug statements the item
> > names are being retrieved correctly, but the attributes returned are
> > those of other items in the queue - not its own item.  however, the
> > model variable is not a global variable... so i'm not sure what's
> > wrong.
>
> > i've declared a bunch of workerthreads(100) and a queue into which
> > new requests are inserted, like so:
>
> > queue = Queue.Queue(0)
> >  WORKERS=100
> > for i in range(WORKERS):
> >    thread = SDBThread(queue)
> >    thread.setDaemon(True)
> >    thread.start()
>
> > the thread:
>
> > class SimpleDBThread ( threading.Thread ):
> >    def __init__ ( self, queue ):
> >            self.__queue = queue
> >            threading.Thread.__init__ ( self )
> >    def run ( self ):
> >            while 1:
> >                    item = self.__queue.get()
> >                    if item!=None:
> >                            model = domain.get_item(item[0])
> >                            logger.debug('sdbthread item:'+item[0])
> >                            title = model['title']
> >                            scraped = model['scraped']
> >                            logger.debug("sdbthread title:"+title)
>
> > any suggestions?
> > thanks
>
>    Hm.  We don't have enough code here to see what's wrong.
> For one thing, we're not seeing how items get put on the queue.  The
> trouble might be at the "put" end.
>
>    Make sure that "model", "item", "title", and "scraped" are not globals.
> Remember, any assignment to them in a global context makes them a global.
>
>    You should never get "None" from the queue unless you put a "None"
> on the queue.  "get()" blocks until there's work to do.
>
>                                         John Nagle

thanks John, Gabriel,
  here's the 'put' side of the requests:

def prepSDBSearch(results):
	modelList = [0]
	counter=1
	for result in results:
		data = [result.item, counter, modelList]
		queue.put(data)
		counter+=1
	while modelList[0] < len(results):
		print 'waiting...'#wait for them to come home
	modelList.pop(0)#now remove '0'
	return modelList

responses to your follow ups:
1)  'item' in the threads is a list that corresponds to the 'data'
list in the above function.  it's not global, and the initial values
seem ok, but i'm not sure if every time i pass in data to the queue it
passes in the same memory address or declares a new 'data' list (which
I guess is what I want)
2)  john, i don't think any of the variables you mentioned are
global.  the 'none' check was just for extra safety.
3)  the first item in the modelList is a counter that keeps track of
the number of threads for this call that have completed - is there any
better way of doing this?

thanks again