[Pythonmac-SIG] Python threading.Thread vs PyObjC performOnMainThread

[gandreas@gandreas.com]

On May 16, 2005, at 1:22 PM, Bob Ippolito wrote:

>
> On May 16, 2005, at 2:01 PM, gandreas at gandreas.com wrote:
>>>
>>
>> The debugged app works like that, but the way that the "remote
>> object" interface works is also designed to use sync TCP on the host
>> side. Since it's pure python, I didn't want to start adding in
>> NSFileHandles, and the obvious solution was "just use threads", and
>> the whole "block on reads" model was much cleaner than any sort of
>> polling.
>>
>
> This isn't a very good argument since you're already using plenty  
> of Objective-C code in there.
>

My goal was to try to not make it worse... (since part of the goal  
was to be able to provide a debugging stub that could be included in  
anything that embedded python, regardless if it was PyObjC based or  
not - or even Mac OS X based).

> Anyway, there's Twisted, asyncore/medusa, etc. for non-blocking  
> Python IO... but NSFileHandle or the like is going to be the  
> simplest thing that's going to work.
>

The more I investigate this, the more this seems to be the path of  
least resistance...

>
>>> If you need a way to interrupt the debugged app asynchronously
>>> (i.e. ctrl-C in gdb), then set up a signal handler in the debugged
>>> app that installs a trace hook from the debugging bootstrap code
>>> that makes it go into debugging mode and send it a signal from the
>>> host app... but that's not something that's ever very clean in  
>>> Python.
>>>
>>
>> I haven't quite figured out a good way to handle that one either, but
>> since the other app is a child, sending a signal should work as good
>> as anything.
>>
>
> There's a "you probably shouldn't use this" function in the C API  
> that lets you write an extension module that sends an exception to  
> another thread, so in theory you could start a background thread  
> that sits on this and waits for a "signal" over TCP or the like  
> from the parent and then uses that to break into the main thread.   
> Don't do this though, there's a lot of reasons not to.
>
> Alternatively, you could start up a second socket on another thread  
> and use it to send a signal to its own pid if it hears from the  
> host.  This would facilitate true remote debugging.
>

I  should be able to directly send the signal from the IDE to the  
targeted app (since it's a child of the IDE), and the standard "catch  
a unix signal invokes the debugger" action should work (since the  
debugger interface on the client talks back to the IDE).

>>>
>>> Update to PyObjC from svn and see if that makes a difference.
>>>
>>
>> Can't do that - the goal is to provide something that, at worse,
>> requires the user to install a given binary package (PyOXIDE is
>> designed to directly call packman if it detects PyObjC not being
>> installed, to make things as transparent as possible).  If I require
>> the user to download the latest sources and compile it, that becomes
>> a nightmare.  If I compile it myself  from "latest beta" and included
>> it with PyOXIDE, that prevents bug fixed releases (since PyOXIDE
>> would be hardcoded to run on the older, buggier, pre-release  
>> version).
>>
>> There's already far too much "well, it will all work great, but first
>> you need to download this (and then compile that, and install this
>> third thing)..." in the MacPython world that it makes me cringe (and
>> certainly the choice of versions of Python included with the OS don't
>> make the problem less either).  Unfortunately, I can't think up a
>> good way to solve that problem so I just cringe a lot and try to not
>> make anything worse...
>>
>
> Well that's too bad.  What you're suggesting will make things much  
> worse.
>
> The Python interpreter used by your application should be private  
> anyway, so you can and should include PyObjC (probably even Python  
> itself) inside your application.

Including an entire "custom" version of Python just seems like a bad  
solution to me.  It may be the only solution, depending on various  
other circumstances, but it seems kind of sad to say "well, OS X  
ships with Python.framework, but it can't be used to write a Python  
IDE".

> I've said this a ton of times, but the ONLY time you should EVER  
> run user Python code from an IDE is from separate processes, unless  
> it's simply a plugin for the IDE.

I personally don't agree with this.  There are plenty of times when I  
just need something to "tinker" with, and the separate process model  
doesn't work well because I find the ability to have a semi- 
persistent state extremely useful (run the script, examine the  
results using an interactive window, manually call some of the  
routines, fix a routine and run that source file again, etc... - not  
to mention that examining the state of objects within the application  
is _much_ faster than having to proxy everything through a socket).   
Now if there were a "persistent external" mode, that would be another  
story (but that's more a "future feature").



> That said, PyObjC 1.3.5 is just about ready to release anyway.
>

Cool!

> If PyOXIDE itself needs a bug fix from a newer version of PyObjC  
> (which in this case, I'm almost 100% positive it does), then you  
> should update PyOXIDE with a new version of PyObjC.
>

I'm leaning towards redoing things to avoid the "here be dragons"  
areas of the map.

>
>> After a good burrito for lunch, I'm thinking that adding an
>> unlockPythonAndPerformSelectorOnMainThread(andRelockAfterwards)
>> method might be the best way to handle this - at least it's something
>> easy that I can add as part of PyOXIDE...
>>
>
> NO!  That is really really horribly bad to do.  Working around a  
> bug in PyObjC by writing code that will break in correct versions  
> of PyObjC is absolutely horrible.  You should NEVER release the  
> lock unless you own it.
>

Here, then, is probably the heart of the problem - the interactions  
between python threads spawned by threading.Thread and the main  
(objc) thread and performSelectorOnMainThread are "undefined", and  
the more I look at it, the more it appears that using  
threading.Thread in a PyObjC app is a bad idea and should probably  
just be chalked up as "unsupported and probably won't work".  It  
appears that I'd need to build something on top of NSThread (in  
Objective-C) with explicit thread state handling to get this to work  
consistently and correctly.



Glenn Andreas                      gandreas at gandreas.com
  <http://www.gandreas.com/> oh my!
quadrium | build, mutate, evolve | images, textures, backgrounds, art