MacOS 10.9.2: threading error using python.org 2.7.6 distribution

[Matthew Pounsett]

I've run into a threading error in some code when I run it on MacOS that works flawlessly on a *BSD system running the same version of python.  I'm running the python 2.7.6 for MacOS distribution from python.org's downloads page.

I have tried to reproduce the error with a simple example, but so far haven't been able to find the element or my code that triggers the error.  I'm hoping someone can suggest some things to try and/or look at.  Googling for "pyton" and the error returns exactly two pages, neither of which are any help.

When I run it through the debugger, I'm getting the following from inside threading.start().  python fails to provide a stack trace when I step into _start_new_thread(), which is a pointer to thread.start_new_thread().  It looks like threading.__bootstrap_inner() may be throwing an exception which thread.start_new_thread() is unable to handle, and for some reason the stack is missing so I get no stack trace explaining the error.

It looks like thread.start_new_thread() is in the binary object, so I can't actually step into it and find where the error is occurring.

> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py(745)start()
-> _start_new_thread(self.__bootstrap, ())
(Pdb) s
> /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py(750)start()
-> self.__started.wait()
(Pdb) Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from
Warning: No stack to get attribute from

My test code (which works) follows the exact same structure as the failing code, making the same calls to the threading module's objects' methods:

----
import threading

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        print "MyThread runs and exits."

def main():
    try:
        t = MyThread()
        t.start()
    except Exception as e:
        print "Failed with {!r}".format(e)

if __name__ == '__main__':
    main()
----

The actual thread object that's failing looks like this:

class RTF2TXT(threading.Thread):
    """
    Takes a directory path and a Queue as arguments.  The directory should be
    a collection of RTF files, which will be read one-by-one, converted to
    text, and each output line will be appended in order to the Queue.
    """
    def __init__(self, path, queue):
        threading.Thread.__init__(self)
        self.path = path
        self.queue = queue

    def run(self):
        logger = logging.getLogger('RTF2TXT')
        if not os.path.isdir(self.path):
            raise TypeError, "supplied path must be a directory"
        for f in sorted(os.listdir(self.path)):
            ff = os.path.join(self.path, f)
            args = [ UNRTF_BIN, '-P', '.', '-t', 'unrtf.text',  ff ]
            logger.debug("Processing file {} with args {!r}".format(f, args))
            p1 = subprocess.Popen( args, stdout=subprocess.PIPE,
                    universal_newlines=True)
            output = p1.communicate()[0]
            try:
                output = output.decode('utf-8', 'ignore')
            except Exception as err:
                logger.error("Failed to decode output: {}".format(err))
                logger.error("Output was: {!r}".format(output))

            for line in output.split("\n"):
                line = line.strip()
                self.queue.put(line)
            self.queue.put("<EOF>")

Note: I only run one instance of this thread.  The Queue object is used to pass work off to another thread for later processing.

If I insert that object into the test code and run it instead of MyThread(), I get the error.  I can't see anything in there that should cause problems for the threading module though... especially since this runs fine on another system with the same version of python.

Any thoughts on what's going on here?