Thread Question

[Ritesh Raj Sarraf]

Bryan Olson on Saturday 05 Aug 2006 13:31 wrote:

>> Exactly.  Only one thread can hold a lock at a time.
> 
> In the code above, a form called a "critical section", we might
> think of a thread as holding the lock when it is between the
> acquire() and release(). But that's not really how Python's
> locks work. A lock, even in the locked state, is not held by
> any particular thread.
> 
>> If a thread tries
>> to acquire a lock that some other thread has, it'll wait until the
>> other thread releases it.
> 
> More accurate: If a thread tries to acquire a lock that is in
> the locked state, it will wait until some thread releases it.
> (Unless it set the blocking flag false.) If more that one thread
> is waiting to acquire the lock, it may be blocked longer.
> 
> I think the doc for threading.Lock is good:
> 
> http://docs.python.org/lib/lock-objects.html
> 

You're correct.
I noticed that even though while one thread acquires the lock, the other threads
don't respect the lock. In fact they just go ahead and execute the statements
within the lock acquire statement. With this behavior, I'm ending up having a
partially corrupted zip archive file.

def run(request, response, func=copy_first_match):
            '''Get items from the request Queue, process them
            with func(), put the results along with the
            Thread's name into the response Queue.
            
            Stop running once an item is None.'''
        
            name = threading.currentThread().getName()
            ziplock = threading.Lock()
            while 1:
                item = request.get()
                if item is None:
                    break
                (sUrl, sFile, download_size, checksum) = stripper(item)
                response.put((name, sUrl, sFile, func(cache, sFile, sSourceDir,
checksum)))
                
                # This will take care of making sure that if downloaded, they
are zipped
                (thread_name, Url, File, exit_status) = responseQueue.get()
                if exit_status == False:
                    log.verbose("%s not available in local cache %s\n" % (File,
cache))
                    if download_from_web(sUrl, sFile, sSourceDir, checksum) !=
True:
                        log.verbose("%s not downloaded from %s and NA in local
cache %s\n\n" % (sFile, sUrl, sRepository))
                    else:
                        # We need this because we can't do join or exists
operation on None
                        if cache is None or os.path.exists(os.path.join(cache,
sFile)):
                            #INFO: The file is already there.
                            pass
                        else:
                            shutil.copy(sFile, cache)
                            if zip_bool:
                                ziplock.acquire()
                                try:
                                    compress_the_file(zip_type_file, sFile,
sSourceDir)
                                    os.remove(sFile) # Remove it because we
don't need the file once it is zipped.
                                finally:
                                    ziplock.release()
                elif exit_status == True:
                    if zip_bool:
                        ziplock.acquire()
                        try:
                            compress_the_file(zip_type_file, sFile, sSourceDir)
                            os.unlink(sFile)
                        finally:
                            ziplock.release()
                    
-- 
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
"Stealing logic from one person is plagiarism, stealing from many is research."
"The great are those who achieve the impossible, the petty are those who
cannot - rrs"