From Steffen.Guhlemann at mailbox.tu-dresden.de Wed Mar 2 19:16:01 2005 From: Steffen.Guhlemann at mailbox.tu-dresden.de (Steffen Guhlemann) Date: Wed Mar 2 19:31:40 2005 Subject: [Python.NET] Bug with System.Diagnostics.Process.ProcessName and tcl Message-ID: <1109787361.422602e1ca1b3@rmc60-231.urz.tu-dresden.de> Hi, I experienced some very strange bugs. Can someone help me? The affected Programm is mainly written in Python with a Tcl-Gui. Some part of the application logic lies in a c#-dll. Deep in this dll a function searches for a certain Process by its id: Process p=Process.GetProcessById(pid); Afterwards the function checks the name of this Process: string name=p.ProcessName; This line makes Python go crazy. At least the Tcl-Gui and the float-Arithmetics (i don't know which is a consequece of which) - we don't know what else is affected. The Tcl-Windows appear with minimal size (0,0) instead of the size they where meant to. Others with certain controls (e.g. a tree-Control) don't even appear. Instead Python throws the following Exception: Traceback (most recent call last): File "C:\tmp\testProcesses\bin\Release\test.py", line 577, in ? width=300) File "C:\tmp\testProcesses\bin\Release\test.py", line 388, in __init__ self.move_cursor(self.root) File "C:\tmp\testProcesses\bin\Release\test.py", line 474, in move_cursor self.see(node.symbol, node.label) File "C:\tmp\testProcesses\bin\Release\test.py", line 446, in see while x2 > self.canvasx(0)+self.winfo_width(): File "C:\Python23\lib\lib-tk\Tkinter.py", line 2028, in canvasx return getdouble(self.tk.call( ValueError: invalid literal for float(): 6.0 The float arithmetic error, we found is not deterministic...if i edit and start the script in IDLE, which also starts the Tree-Control mentioned above, then somewhen it starts to go wrong with these lines: val1=0.95 val2=1024 val3=int(val1*val2) print val3 If beforehand of these lines no c#-function is called that uses some ProcessName, the result is 972, otherwise the result could be 0. This is not deterministic, the Tcl-Problem is deterministic. Thanks Steffen From ahaffner at lexarmedia.com Fri Mar 4 01:42:30 2005 From: ahaffner at lexarmedia.com (Alan Haffner) Date: Fri Mar 4 03:33:40 2005 Subject: [Python.NET] Using PythonNet to import a .Net compiled C++ dll Message-ID: <10BCECC413264D4D8E8E2A9515229E45C508B2@phoenix.lexarmedia.com> Hello, Is it possible to import a Managed .Net C++ DLL file? I seem to be able to import C# DLLs and the namespace of a C++ DLL, but not its Public Class. Thanks -Alan Haffner -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/pythondotnet/attachments/20050303/472ec72d/attachment.html From alanhaffner at yahoo.com Fri Mar 4 01:45:57 2005 From: alanhaffner at yahoo.com (alan haffner) Date: Fri Mar 4 03:33:44 2005 Subject: [Python.NET] Importing a Managed .Net C++ DLL Message-ID: <20050304004557.95551.qmail@web81209.mail.yahoo.com> Hello, Is it possible to import a Managed .Net C++ DLL file? I seem to be able to import C# DLLs and the namespace of a C++ DLL, but not its Public Class. Thanks -Alan Haffner From brian at zope.com Fri Mar 4 03:35:58 2005 From: brian at zope.com (Brian Lloyd) Date: Fri Mar 4 03:36:06 2005 Subject: [Python.NET] Importing a Managed .Net C++ DLL In-Reply-To: <20050304004557.95551.qmail@web81209.mail.yahoo.com> Message-ID: <20050304023600.169FE3B805B@smtp.zope.com> > Is it possible to import a Managed .Net C++ DLL file? > I seem to be able to import C# DLLs and the namespace > of a C++ DLL, but not its Public Class. It should be possible to see and use anything that Reflection can see, regardless of the original src language. It would be interesting to see if you are able to reflect on the types in your MC++ dll... Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From brian at zope.com Wed Mar 9 15:59:08 2005 From: brian at zope.com (Brian Lloyd) Date: Wed Mar 9 15:59:47 2005 Subject: [Python.NET] Bug with System.Diagnostics.Process.ProcessName and tcl In-Reply-To: <1109787361.422602e1ca1b3@rmc60-231.urz.tu-dresden.de> Message-ID: This has all the symptoms of some kind of memory error - unfortunately the symptoms are often not directly helpful in diagnosing the root the problem (since the memory error occurred sometime before the random insanity started...) I'm not up on the implementation details of tcl/tk, but its not out of the question that it could interact badly with objects that come from PythonNet (which has to do some arguably evil things with object structs, etc.) If you can narrow it down to the simplest possible case and send me that script, I can try to help investigate... Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com > -----Original Message----- > From: pythondotnet-bounces+brian=zope.com@python.org > [mailto:pythondotnet-bounces+brian=zope.com@python.org]On Behalf Of > Steffen Guhlemann > Sent: Wednesday, March 02, 2005 1:16 PM > To: pythondotnet@python.org > Subject: [Python.NET] Bug with System.Diagnostics.Process.ProcessName > and tcl > > > Hi, > > I experienced some very strange bugs. Can someone help me? > > The affected Programm is mainly written in Python with a Tcl-Gui. > Some part of > the application logic lies in a c#-dll. Deep in this dll a > function searches > for a certain Process by its id: > > Process p=Process.GetProcessById(pid); > > Afterwards the function checks the name of this Process: > > string name=p.ProcessName; > > This line makes Python go crazy. At least the Tcl-Gui and the > float-Arithmetics > (i don't know which is a consequece of which) - we don't know what else is > affected. > > The Tcl-Windows appear with minimal size (0,0) instead of the > size they where > meant to. Others with certain controls (e.g. a tree-Control) > don't even appear. > Instead Python throws the following Exception: > > Traceback (most recent call last): > File "C:\tmp\testProcesses\bin\Release\test.py", line 577, in ? > width=300) > File "C:\tmp\testProcesses\bin\Release\test.py", line 388, in __init__ > self.move_cursor(self.root) > File "C:\tmp\testProcesses\bin\Release\test.py", line 474, in > move_cursor > self.see(node.symbol, node.label) > File "C:\tmp\testProcesses\bin\Release\test.py", line 446, in see > while x2 > self.canvasx(0)+self.winfo_width(): > File "C:\Python23\lib\lib-tk\Tkinter.py", line 2028, in canvasx > return getdouble(self.tk.call( > ValueError: invalid literal for float(): 6.0 > > The float arithmetic error, we found is not deterministic...if i > edit and start > the script in IDLE, which also starts the Tree-Control mentioned > above, then > somewhen it starts to go wrong with these lines: > > val1=0.95 > val2=1024 > val3=int(val1*val2) > print val3 > > If beforehand of these lines no c#-function is called that uses some > ProcessName, the result is 972, otherwise the result could be 0. > This is not > deterministic, the Tcl-Problem is deterministic. > > Thanks > > Steffen > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From roman.yakovenko at gmail.com Thu Mar 10 12:41:19 2005 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Thu Mar 10 12:41:21 2005 Subject: [Python.NET] Importing a Managed .Net C++ DLL In-Reply-To: <20050304023600.169FE3B805B@smtp.zope.com> References: <20050304004557.95551.qmail@web81209.mail.yahoo.com> <20050304023600.169FE3B805B@smtp.zope.com> Message-ID: <7465b6170503100341781aca3a@mail.gmail.com> I use Python.Net to test dll written in manage C++. ( I have more than 270 test cases ) It is not really matter in wich language you write your dll. You can get full access to all methods and classes defined as public within dll. Also reflection works as expected Roman On Thu, 3 Mar 2005 21:35:58 -0500, Brian Lloyd wrote: > > Is it possible to import a Managed .Net C++ DLL file? > > I seem to be able to import C# DLLs and the namespace > > of a C++ DLL, but not its Public Class. > > It should be possible to see and use anything that Reflection > can see, regardless of the original src language. It would be > interesting to see if you are able to reflect on the types in > your MC++ dll... > > > Brian Lloyd brian@zope.com > V.P. Engineering 540.361.1716 > Zope Corporation http://www.zope.com > > > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From roman.yakovenko at gmail.com Mon Mar 14 07:23:39 2005 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Mon Mar 14 07:23:44 2005 Subject: [Python.NET] AcquireThreadLock - bug? future? Message-ID: <7465b61705031322237e4cfac1@mail.gmail.com> Hi. I have small problem with multithreading and need help to solve it. I have dll written in managed C++. Within dll I have class that has static function. I'd like to call this function from a few threads from python. I failed to do this. Calling function staled and this is a stack dump from debugger ( of my program ) python.runtime.dll!Python.Runtime.PythonEngine.AcquireThreadLock() + 0x40 bytes python.runtime.dll!Python.Runtime.MethodBinder.Invoke(int inst = 11793552, int args = 9900080, int kw = 0) + 0xf0 bytes python.runtime.dll!Python.Runtime.MethodObject.Invoke(int target = 11793552, int args = 9900080, int kw = 0) + 0x25 bytes python.runtime.dll!Python.Runtime.MethodBinding.tp_call(int ob = 11794928, int args = 9900080, int kw = 0) + 0x137 bytes I don't understand: 1. Why Python.Net tries to acquire thread lock, while calling function ? I am possible to call get property from a few threads. From managed C++ point of view ( I could be wrong here ) getting value of property it is just syntax sugaring for get_??? call. I want to say that if I able to call "get property" from a few threads I should be able to call function from a few threads. 2. Why at all Python.Net tries to synchronize function calls? I can modify and compile source code of Python.Net, but I know to little about reasons of calling AcquireThreadLock. Thanks for help Roman From brian at zope.com Mon Mar 14 13:07:31 2005 From: brian at zope.com (Brian Lloyd) Date: Mon Mar 14 13:07:37 2005 Subject: [Python.NET] AcquireThreadLock - bug? future? In-Reply-To: <7465b61705031322237e4cfac1@mail.gmail.com> Message-ID: Essentially the engine needs to own the thread lock in order to call Python C API functions. When a (managed) method is called, the MethodBinder: - releases the lock (so that other threads could potentially run) - calls the managed method - re-acquires the lock after the call (because it needs to use the C API again to Pythonify the result, etc.) So it sounds like some thread is unable to re-acquire the lock. This gets tricky _really_ quickly, because you can actually be calling a managed object that may later call back into Python code... etc. Note that this doesn't happen for properties, because I arbitrarily decided that since properties are expected to to be lightweight that it wasn't worth the overhead ;) Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com > -----Original Message----- > From: pythondotnet-bounces@python.org > [mailto:pythondotnet-bounces@python.org]On Behalf Of Roman Yakovenko > Sent: Monday, March 14, 2005 1:24 AM > To: pythondotnet@python.org > Subject: [Python.NET] AcquireThreadLock - bug? future? > > > Hi. I have small problem with multithreading and need help to solve it. > I have dll written in managed C++. Within dll I have class that has > static function. I'd like to call this function from a few threads > from python. I failed to do this. Calling function staled and this is > a stack dump from debugger ( of my program ) > > python.runtime.dll!Python.Runtime.PythonEngine.AcquireThreadLock() > + 0x40 bytes > > python.runtime.dll!Python.Runtime.MethodBinder.Invoke(int inst = > 11793552, int args = 9900080, int kw = 0) + 0xf0 bytes > > python.runtime.dll!Python.Runtime.MethodObject.Invoke(int target = > 11793552, int args = 9900080, int kw = 0) + 0x25 bytes > > python.runtime.dll!Python.Runtime.MethodBinding.tp_call(int ob = > 11794928, int args = 9900080, int kw = 0) + 0x137 bytes > > I don't understand: > 1. Why Python.Net tries to acquire thread lock, while calling > function ? > I am possible to call get property from a few threads. From > managed C++ point of view ( I could be wrong here ) getting value of > property it is just syntax sugaring for get_??? call. I want to say > that if I able to call "get property" from a few threads I should be > able to call function from a few threads. > 2. Why at all Python.Net tries to synchronize function calls? > > I can modify and compile source code of Python.Net, but I know to > little about reasons > of calling AcquireThreadLock. > > Thanks for help > > Roman > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From bksasha at zahav.net.il Mon Mar 14 17:00:40 2005 From: bksasha at zahav.net.il (bksasha@zahav.net.il) Date: Mon Mar 14 13:28:36 2005 Subject: [Python.NET] AcquireThreadLock - bug? future? References: Message-ID: <01f601c528ae$fc1089d0$fd01a8c0@gate> Greetings I think this may be tye reason why I can successfuly cal any property from Zope, but any function call except constructor hangs the instance Can there be some option to treat methods like properties also, so product become usable from Zope? Thanks Alexander ----- Original Message ----- From: "Brian Lloyd" To: "Roman Yakovenko" ; Sent: Monday, March 14, 2005 2:07 PM Subject: RE: [Python.NET] AcquireThreadLock - bug? future? > Essentially the engine needs to own the thread lock in > order to call Python C API functions. When a (managed) > method is called, the MethodBinder: > > - releases the lock (so that other threads could > potentially run) > > - calls the managed method > > - re-acquires the lock after the call (because it > needs to use the C API again to Pythonify the > result, etc.) > > So it sounds like some thread is unable to re-acquire > the lock. This gets tricky _really_ quickly, because > you can actually be calling a managed object that may > later call back into Python code... etc. > > Note that this doesn't happen for properties, because I > arbitrarily decided that since properties are expected to > to be lightweight that it wasn't worth the overhead ;) > > > Brian Lloyd brian@zope.com > V.P. Engineering 540.361.1716 > Zope Corporation http://www.zope.com > > > > -----Original Message----- > > From: pythondotnet-bounces@python.org > > [mailto:pythondotnet-bounces@python.org]On Behalf Of Roman Yakovenko > > Sent: Monday, March 14, 2005 1:24 AM > > To: pythondotnet@python.org > > Subject: [Python.NET] AcquireThreadLock - bug? future? > > > > > > Hi. I have small problem with multithreading and need help to solve it. > > I have dll written in managed C++. Within dll I have class that has > > static function. I'd like to call this function from a few threads > > from python. I failed to do this. Calling function staled and this is > > a stack dump from debugger ( of my program ) > > > > python.runtime.dll!Python.Runtime.PythonEngine.AcquireThreadLock() > > + 0x40 bytes > > > > python.runtime.dll!Python.Runtime.MethodBinder.Invoke(int inst = > > 11793552, int args = 9900080, int kw = 0) + 0xf0 bytes > > > > python.runtime.dll!Python.Runtime.MethodObject.Invoke(int target = > > 11793552, int args = 9900080, int kw = 0) + 0x25 bytes > > > > python.runtime.dll!Python.Runtime.MethodBinding.tp_call(int ob = > > 11794928, int args = 9900080, int kw = 0) + 0x137 bytes > > > > I don't understand: > > 1. Why Python.Net tries to acquire thread lock, while calling > > function ? > > I am possible to call get property from a few threads. From > > managed C++ point of view ( I could be wrong here ) getting value of > > property it is just syntax sugaring for get_??? call. I want to say > > that if I able to call "get property" from a few threads I should be > > able to call function from a few threads. > > 2. Why at all Python.Net tries to synchronize function calls? > > > > I can modify and compile source code of Python.Net, but I know to > > little about reasons > > of calling AcquireThreadLock. > > > > Thanks for help > > > > Roman > > _________________________________________________ > > Python.NET mailing list - PythonDotNet@python.org > > http://mail.python.org/mailman/listinfo/pythondotnet > > > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet From roman.yakovenko at gmail.com Mon Mar 14 14:08:19 2005 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Mon Mar 14 14:08:22 2005 Subject: [Python.NET] AcquireThreadLock - bug? future? In-Reply-To: References: <7465b61705031322237e4cfac1@mail.gmail.com> Message-ID: <7465b6170503140508f86871e@mail.gmail.com> On Mon, 14 Mar 2005 07:07:31 -0500, Brian Lloyd wrote: > Essentially the engine needs to own the thread lock in > order to call Python C API functions. When a (managed) > method is called, the MethodBinder: > > - releases the lock (so that other threads could > potentially run) > > - calls the managed method > > - re-acquires the lock after the call (because it > needs to use the C API again to Pythonify the > result, etc.) > > So it sounds like some thread is unable to re-acquire > the lock. This gets tricky _really_ quickly, because > you can actually be calling a managed object that may > later call back into Python code... etc. > > Note that this doesn't happen for properties, because I > arbitrarily decided that since properties are expected to > to be lightweight that it wasn't worth the overhead ;) I never wrote programs using Python C API. I think this is the first reason why I don't understand why I should acquire thread lock Pythonify the result. I definitely will read python manuals. An other thing that I understand from your answer is that if I wrote property that takes time ( for example server.IsConnected - makes ping to the remote server ) I will run into some undefined behaviour ? I don't have enough knowledge. Could you propose some solution ? If you need more information I can debug application. > > Brian Lloyd brian@zope.com > V.P. Engineering 540.361.1716 > Zope Corporation http://www.zope.com > > > > -----Original Message----- > > From: pythondotnet-bounces@python.org > > [mailto:pythondotnet-bounces@python.org]On Behalf Of Roman Yakovenko > > Sent: Monday, March 14, 2005 1:24 AM > > To: pythondotnet@python.org > > Subject: [Python.NET] AcquireThreadLock - bug? future? > > > > > > Hi. I have small problem with multithreading and need help to solve it. > > I have dll written in managed C++. Within dll I have class that has > > static function. I'd like to call this function from a few threads > > from python. I failed to do this. Calling function staled and this is > > a stack dump from debugger ( of my program ) > > > > python.runtime.dll!Python.Runtime.PythonEngine.AcquireThreadLock() > > + 0x40 bytes > > > > python.runtime.dll!Python.Runtime.MethodBinder.Invoke(int inst = > > 11793552, int args = 9900080, int kw = 0) + 0xf0 bytes > > > > python.runtime.dll!Python.Runtime.MethodObject.Invoke(int target = > > 11793552, int args = 9900080, int kw = 0) + 0x25 bytes > > > > python.runtime.dll!Python.Runtime.MethodBinding.tp_call(int ob = > > 11794928, int args = 9900080, int kw = 0) + 0x137 bytes > > > > I don't understand: > > 1. Why Python.Net tries to acquire thread lock, while calling > > function ? > > I am possible to call get property from a few threads. From > > managed C++ point of view ( I could be wrong here ) getting value of > > property it is just syntax sugaring for get_??? call. I want to say > > that if I able to call "get property" from a few threads I should be > > able to call function from a few threads. > > 2. Why at all Python.Net tries to synchronize function calls? > > > > I can modify and compile source code of Python.Net, but I know to > > little about reasons > > of calling AcquireThreadLock. > > > > Thanks for help > > > > Roman > > _________________________________________________ > > Python.NET mailing list - PythonDotNet@python.org > > http://mail.python.org/mailman/listinfo/pythondotnet > > > From borice23 at yahoo.com Sat Mar 19 18:30:32 2005 From: borice23 at yahoo.com (Boris Capitanu) Date: Sat Mar 19 18:30:36 2005 Subject: [Python.NET] PythonEngine.UnloadModule? Message-ID: <20050319173032.38203.qmail@web51808.mail.yahoo.com> Hello, Is there any way to un-import a module? Suppose I create a plug-in scenario and load plugins with ImportModule... at some point I want to unload a plugin, so something like PythonEngine.UnloadModule is needed. Is this possible to do? Thank you! Boris Capitanu capitanu@uiuc.edu __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From borice23 at yahoo.com Sat Mar 19 18:32:41 2005 From: borice23 at yahoo.com (Boris Capitanu) Date: Sat Mar 19 18:32:44 2005 Subject: [Python.NET] Python bridge for .NET CF Message-ID: <20050319173241.81212.qmail@web51802.mail.yahoo.com> Hello again, I am also interested if there is a way to get the bridge to work with Python for Pocket PCs. There is a Python 2.2 available for Pocket PCs from http://www.murkworks.com/Research/Python/PocketPCPython/Overview Thank you, Boris Capitanu capitanu@uiuc.edu __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From stan at phidani.be Mon Mar 21 17:30:33 2005 From: stan at phidani.be (Stan Pinte) Date: Mon Mar 21 17:30:43 2005 Subject: [Python.NET] bug in managed object usage handling within a module. Message-ID: <423EF6A9.80300@phidani.be> hello, I will try to describe my problem in an abstract way... I have the following C# objects: namespace Foo { abstract class A class B : A } namespace Bar { class C { static object createInstance(A argument) { ... } } } when I do the following in a module located below the current directory (example: functional/testfoobar.py) , it gives me an error: import sys sys.path.append('bin') rom CLR.System.Reflection import Assembly Assembly.LoadWithPartialName("foobar") from CLR.Bar import C from CLR.Foo import A,B x = B() assert isinstance(x, A) h = C.createInstance(x) --> TypeError: no method matches given arguments while if I do it in a module located in the current directory (the top level of the sources), it works all right. What is strange is that assert isinstance(x, A) works all right in both cases, it just fails on the last line...and it properly recognize static method of class C... If really needed, I can try to engineer a compilable test case... it is some work though... let me know, thanks, Stan. From brian at zope.com Mon Mar 21 22:22:09 2005 From: brian at zope.com (Brian Lloyd) Date: Mon Mar 21 22:22:25 2005 Subject: [Python.NET] Python bridge for .NET CF In-Reply-To: <20050319173241.81212.qmail@web51802.mail.yahoo.com> Message-ID: > I am also interested if there is a way to get the > bridge to work with Python for Pocket PCs. There is a > Python 2.2 available for Pocket PCs from > http://www.murkworks.com/Research/Python/PocketPCPython/Overview > > Thank you, > > Boris Capitanu Hi Boris - I have a feeling this would be a tough job. The current version of Python for .NET is based on Python2.3+, and there were some non-trivial changes to type structures, etc. in CPython from 2.2 to 2.3. You could possibly use the old preview-2 release (which was based on 2.2). I'm also not sure that the .NET CF supports all of the reflection-related things that Python for .NET needs to work. Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From stan at phidani.be Tue Mar 22 13:26:53 2005 From: stan at phidani.be (Stan Pinte) Date: Tue Mar 22 13:26:58 2005 Subject: [Python.NET] from CLR.mynamespace import * doesn't work Message-ID: <42400F0D.9060603@phidani.be> hello, from CLR.mynamespace import * doesn't work. I have to import every class manually. Any hint? Note: would'it be worth opening a bugzilla-like system for pythondotnet? thanks a lot, Stan. From brian at zope.com Tue Mar 22 15:54:48 2005 From: brian at zope.com (Brian Lloyd) Date: Tue Mar 22 15:55:02 2005 Subject: [Python.NET] from CLR.mynamespace import * doesn't work In-Reply-To: <42400F0D.9060603@phidani.be> Message-ID: > hello, > > from CLR.mynamespace import * doesn't work. > > I have to import every class manually. > > Any hint? This purposely doesn't work, as it would impose a pretty big performance hit and a lot of extra complexity in the PythonNet runtime. I just noticed that I hadn't documented that in the README though - I've updated it for the next release. If this is a significant issue for enough people, it might be worth revisiting, but as I recall supporting the '*' form introduces a lot of strange and unexpected side effects with regard to module loading, since an assembly can contribute names to multiple 'modules' from Python's point of view, unlike pure Python modules. > Note: would'it be worth opening a bugzilla-like system for pythondotnet? There is a bug tracker at: http://www.zope.org/Members/Brian/PythonNet/Collector/ Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From stan at phidani.be Wed Mar 23 08:57:00 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 08:57:01 2005 Subject: [Python.NET] (no subject) Message-ID: <4241214C.309@phidani.be> hello, I am running Simpy (python simulation framework) within pythondotnet, and, even though this process is single-thread, it hangs misteriously, in an unpredictable way... Python console cease to respond to Ctrl-C events... Here is the current Thread status: Thread Start Address: Symbol Name: Line Number: PC: mscoree!_CorExeMain() + 0x0 --- 7917D08C Thread Stack: ntdll ! KiFastSystemCallRet() + 0x KERNEL32 ! WaitForSingleObject() + 0x12 python24 ! PySys_WriteStderr() + 0x14d python24 ! PyTuple_Type() + 0x0 as the entry point in the hanging thread (higher on stack) is PyTuple_Type() and as PyTuple is defined in C# (src/runtime/PyTuple.cs), I suspect this might be the cause of my problem. [PythonNet-1.0-beta4]> grep -nr "PyTuple_Type" . Binary file ./DLLs/_socket.pyd matches Binary file ./python24.dll matches [PythonNet-1.0-beta4]> However, I would like to be able to go higher in the stack, to see what caused this deadlock. Any proposed strategy to guess what happened, or to track down the problem? thanks a lot, Stan. From stan at phidani.be Wed Mar 23 08:57:25 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 08:57:27 2005 Subject: [Python.NET] deadlock Message-ID: <42412165.9090500@phidani.be> hello, I am running Simpy (python simulation framework) within pythondotnet, and, even though this process is single-thread, it hangs misteriously, in an unpredictable way... Python console cease to respond to Ctrl-C events... Here is the current Thread status: Thread Start Address: Symbol Name: Line Number: PC: mscoree!_CorExeMain() + 0x0 --- 7917D08C Thread Stack: ntdll ! KiFastSystemCallRet() + 0x KERNEL32 ! WaitForSingleObject() + 0x12 python24 ! PySys_WriteStderr() + 0x14d python24 ! PyTuple_Type() + 0x0 as the entry point in the hanging thread (higher on stack) is PyTuple_Type() and as PyTuple is defined in C# (src/runtime/PyTuple.cs), I suspect this might be the cause of my problem. [PythonNet-1.0-beta4]> grep -nr "PyTuple_Type" . Binary file ./DLLs/_socket.pyd matches Binary file ./python24.dll matches [PythonNet-1.0-beta4]> However, I would like to be able to go higher in the stack, to see what caused this deadlock. Any proposed strategy to guess what happened, or to track down the problem? thanks a lot, Stan. From stan at phidani.be Wed Mar 23 11:03:46 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 11:04:22 2005 Subject: [Python.NET] deadlock In-Reply-To: <42412165.9090500@phidani.be> References: <42412165.9090500@phidani.be> Message-ID: <42413F02.6080601@phidani.be> Stan Pinte a ?crit : >hello, > >I am running Simpy (python simulation framework) within pythondotnet, >and, even though this process is single-thread, it hangs misteriously, >in an unpredictable way... > >Python console cease to respond to Ctrl-C events... > >Here is the current Thread status: > >Thread Start Address: >Symbol Name: Line Number: PC: >mscoree!_CorExeMain() + 0x0 --- 7917D08C > >Thread Stack: > >ntdll ! KiFastSystemCallRet() + 0x >KERNEL32 ! WaitForSingleObject() + 0x12 >python24 ! PySys_WriteStderr() + 0x14d >python24 ! PyTuple_Type() + 0x0 > > more info, from windows debugging utility: [Debugging Tools for Windows]> tlist 2632 2632 python.exe OleMainThreadWndName CWD: f:\home\working\phidani\project-stm\tools\ VirtualSize: 171300 KB PeakVirtualSize: 171300 KB WorkingSetSize: 22204 KB PeakWorkingSetSize: 22204 KB NumberOfThreads: 6 2916 Win32StartAddr:0x7917d08c LastErr:0x00000000 State:Waiting 636 Win32StartAddr:0x791d4d45 LastErr:0x00000000 State:Waiting 3796 Win32StartAddr:0x791d0020 LastErr:0x00000000 State:Waiting 2288 Win32StartAddr:0x77e76bf0 LastErr:0x00000000 State:Waiting 2124 Win32StartAddr:0x774fd888 LastErr:0x00000000 State:Waiting 984 Win32StartAddr:0x00000000 LastErr:0x00000000 State:Waiting 1.0.0.0 shp 0x00400000 c:\export\PythonNet-1.0-beta4\python.exe --> so it is multi-threaded, actually. >as the entry point in the hanging thread (higher on stack) is >PyTuple_Type() and as PyTuple is defined in C# (src/runtime/PyTuple.cs), >I suspect this might be the cause of my problem. > >[PythonNet-1.0-beta4]> grep -nr "PyTuple_Type" . >Binary file ./DLLs/_socket.pyd matches >Binary file ./python24.dll matches >[PythonNet-1.0-beta4]> > >However, I would like to be able to go higher in the stack, to see what >caused this deadlock. > >Any proposed strategy to guess what happened, or to track down the problem? > >thanks a lot, > >Stan. > >_________________________________________________ >Python.NET mailing list - PythonDotNet@python.org >http://mail.python.org/mailman/listinfo/pythondotnet > > > > From stan at phidani.be Wed Mar 23 13:18:12 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 13:18:23 2005 Subject: [Python.NET] build a debuggable version of python dot net Message-ID: <42415E84.9090402@phidani.be> hello, when trying to debug a pythondotnet program in WinDbg, it gives me the following error: ModLoad: 1e000000 1e1ca000 C:\export\PythonNet-1.0-beta4\python24.dll HEAP[python.exe]: Invalid Address specified to RtlFreeHeap( 00140000, 071B5994 ) (ecc.cc8): Break instruction exception - code 80000003 (first chance) eax=071b598c ebx=071b598c ecx=7c91eb05 edx=0012e7d6 esi=00140000 edi=071b598c eip=7c901230 esp=0012e9e0 ebp=0012e9e4 iopl=0 nv up ei pl nz na pe nc cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000202 ntdll!DbgBreakPoint: 7c901230 cc int 3 --> Any idea how I can build a debuggable version of pythondotnet? thanks, Stan. From stan at phidani.be Wed Mar 23 14:33:57 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 14:34:04 2005 Subject: [Python.NET] deadlock [More info] In-Reply-To: <42413F02.6080601@phidani.be> References: <42412165.9090500@phidani.be> <42413F02.6080601@phidani.be> Message-ID: <42417045.7050601@phidani.be> Stan Pinte a ?crit : >Stan Pinte a ?crit : > > > >>hello, >> >>I am running Simpy (python simulation framework) within pythondotnet, >>and, even though this process is single-thread, it hangs misteriously, >>in an unpredictable way... >> >>Python console cease to respond to Ctrl-C events... >> >>Here is the current Thread status: >> >>Thread Start Address: >>Symbol Name: Line Number: PC: >>mscoree!_CorExeMain() + 0x0 --- 7917D08C >> >>Thread Stack: >> >>ntdll ! KiFastSystemCallRet() + 0x >>KERNEL32 ! WaitForSingleObject() + 0x12 >>python24 ! PySys_WriteStderr() + 0x14d >>python24 ! PyTuple_Type() + 0x0 >> >> >> hello, actually, the code is always freezing just before a "yield". From stan at phidani.be Wed Mar 23 17:04:58 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 23 17:05:43 2005 Subject: [Python.NET] deadlock [More info] In-Reply-To: <42417045.7050601@phidani.be> References: <42412165.9090500@phidani.be> <42413F02.6080601@phidani.be> <42417045.7050601@phidani.be> Message-ID: <424193AA.3010300@phidani.be> summary info: this was caused by the usage of Queue.queue in my code. I removed all refs to Queue, and am now using a non-reentrant, unsynchronized queue. This solved the problem. I submitted a new bug to sourceforge - python: http://sourceforge.net/tracker/index.php?func=detail&aid=1169108&group_id=5470&atid=105470 Stan. From torgeir.johansen at chello.no Fri Mar 25 12:29:28 2005 From: torgeir.johansen at chello.no (Torgeir Johansen) Date: Fri Mar 25 12:29:29 2005 Subject: [Python.NET] Python 2.4 hang when calling .NET dll function Message-ID: <001301c5312d$e88f6c50$0401a8c0@yoda> I have been able to successfully use my .NET DLL (written in C#) using python 2.4 and the Python for .NET DLLs. I basically have one python module implementing a class that is using methods from my C# DLLs. My test application (singlethreaded) ran great, but when integrating the python module that uses the .NET DLLs in a multithreaded program, one call to a .NET function just hangs. All threads are hanging, and I get no response using CTRL+C. I have debugged my application using ntsd, and I do not see any bad behaviour from my called DLLs. Are there some issues with locking/unlocking mechanism when calling functions from .NET DLLs ? Regards Torgeir From stan at phidani.be Fri Mar 25 13:03:15 2005 From: stan at phidani.be (Stan Pinte) Date: Fri Mar 25 13:03:15 2005 Subject: [Python.NET] Python 2.4 hang when calling .NET dll function In-Reply-To: <001301c5312d$e88f6c50$0401a8c0@yoda> References: <001301c5312d$e88f6c50$0401a8c0@yoda> Message-ID: <4243FE03.3070304@phidani.be> Torgeir Johansen a ?crit : >I have been able to successfully use my .NET DLL (written in C#) using >python 2.4 and the Python for .NET DLLs. I basically have one python module >implementing a class that is using methods from my C# DLLs. > >My test application (singlethreaded) ran great, but when integrating the >python module that uses the .NET DLLs in a multithreaded program, one call >to a .NET function just hangs. All threads are hanging, and I get no >response using CTRL+C. > >I have debugged my application using ntsd, and I do not see any bad >behaviour from my called DLLs. Are there some issues with locking/unlocking >mechanism when calling functions from .NET DLLs ? > > Yes there are...And I don't know if they are due to interaction between python and .net, or only to python. I submitted a bug recently on python.org: http://sourceforge.net/tracker/index.php?func=detail&aid=1169108&group_id=5470&atid=105470 >Regards Torgeir > >_________________________________________________ >Python.NET mailing list - PythonDotNet@python.org >http://mail.python.org/mailman/listinfo/pythondotnet > > > > From brian at zope.com Fri Mar 25 15:35:23 2005 From: brian at zope.com (Brian Lloyd) Date: Fri Mar 25 15:35:42 2005 Subject: [Python.NET] Python 2.4 hang when calling .NET dll function In-Reply-To: <4243FE03.3070304@phidani.be> Message-ID: > >My test application (singlethreaded) ran great, but when integrating the > >python module that uses the .NET DLLs in a multithreaded > program, one call > >to a .NET function just hangs. All threads are hanging, and I get no > >response using CTRL+C. > > > >I have debugged my application using ntsd, and I do not see any bad > >behaviour from my called DLLs. Are there some issues with > locking/unlocking > >mechanism when calling functions from .NET DLLs ? > > > > > > Yes there are...And I don't know if they are due to interaction between > python and .net, or only to python. There is definitely a problem in the way the bridge is handling the GIL (global interpreter lock) in multithreaded situations. I'm working right now on a beta 5 that re-vamps the GIL handling and hopefully takes care of these issues. > I submitted a bug recently on python.org: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1169108&g roup_id=5470&atid=105470 Note that this is the Python bug tracker -- this issue is specific to Python for .NET, not the core CPython so it would be best to use the tracker at http://www.zope.org/Members/Brian/PythonNet/Collector/ Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From torgeir.johansen at chello.no Fri Mar 25 18:19:39 2005 From: torgeir.johansen at chello.no (Torgeir Johansen) Date: Fri Mar 25 18:19:32 2005 Subject: [Python.NET] Python 2.4 hang when calling .NET dll function Message-ID: <001201c5315e$d3e1bac0$fe01a8c0@hackman> Great, Brian. I successfully reproduced the state with a very simple application. Here is very dumb python program that uses the CLR: from CLR.System import String class cSharpCaller( object ): def __init__( self ): print "Created a cSharpCaller instance" self.__cSharpString = None def createCSharpString( self, str ): self.__cSharpString = String( str ) def returnCSharpString( self ): return self.__cSharpString.ToString() And if I test this with this (dumb) threethreaded testprogram: import threading, thread, time from cSharpCaller import cSharpCaller def start_threads(amount=5): for i in range(amount): thread = threading.Thread(target=process_thread ) print "starting thread ", thread.getName() thread.start() def process_thread( ): print "thread started " for i in range( 2 ): print "Hi, I'm a thread" time.sleep( 1 ) cs = cSharpCaller( ) cs.createCSharpString( "created from a thread" ) print "cs.returnCSharpString() = ", cs.returnCSharpString() print "thread %s ended" start_threads( 3 ) I get this output: starting thread thread0 thread started Hi, I'm thread %s starting thread thread1 thread started Hi, I'm thread %s starting thread thread2 thread started Hi, I'm thread %s Hi, I'm thread %s Hi, I'm thread %s Hi, I'm thread %s Created a cSharpCaller instance Created a cSharpCaller instance cs.returnCSharpString() = and I have to CTRL+Pause to get out of the hang. Clearly hangs on the first String.ToString() method call. Regards, Torgeir -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/pythondotnet/attachments/20050325/76c656c3/attachment.html From brian at zope.com Tue Mar 29 04:31:14 2005 From: brian at zope.com (Brian Lloyd) Date: Tue Mar 29 04:31:35 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available Message-ID: Python for .NET 1.0 beta 5 is now available - you can download it from: http://www.zope.org/Members/Brian/PythonNet/ Python for .NET is a near-seamless integration of Python with the .NET common language runtime. For more details, see the README: http://www.zope.org/Members/Brian/PythonNet/README.html The beta 5 release features a refactoring of thread management support, making the runtime stable for multi-threaded applications, as well as several improvements and bug-fixes. Special thanks to all who sent in thread-related scenarios and test cases! I have tried to incorporate as many of these as possible into the unit tests - if you still have problems that you suspect may be related to threading or interpreter lock issues, please let me know asap. There is also now a mailing list for discussion, questions and issues related to Python for .NET at: pythondotnet@python.org. To subscribe to the mailing list or read the online archives, see: http://mail.python.org/mailman/listinfo/pythondotnet Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From stan at phidani.be Wed Mar 30 18:17:16 2005 From: stan at phidani.be (Stan Pinte) Date: Wed Mar 30 18:17:17 2005 Subject: [Python.NET] application crash. Message-ID: <424AD10C.3040500@phidani.be> hello, the following application (python/C#) is working properly, when I do not run simpy, a python framework. Once I enable, I get an immediate crash. See enclosed for runnable/compilable example. (run "python test.py"). To run, it needs the Simpy framework, GPL, that you can get there: http://simpy.sourceforge.net/ Simpy makes heavy uses of the "yield" construct. as soon as I uncomment that line in the script: #simulate() I get an immediate crash. I see no reason for this crash. I will open a tracker item. Stan. From brian at zope.com Wed Mar 30 18:32:36 2005 From: brian at zope.com (Brian Lloyd) Date: Wed Mar 30 18:32:58 2005 Subject: [Python.NET] application crash. In-Reply-To: <424AD10C.3040500@phidani.be> Message-ID: Hi Stan - is this with the most recent (b5) release of Python for .NET? If not, could you try with that and see if you get the same result? Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com > -----Original Message----- > From: pythondotnet-bounces@python.org > [mailto:pythondotnet-bounces@python.org]On Behalf Of Stan Pinte > Sent: Wednesday, March 30, 2005 11:17 AM > To: pythondotnet@python.org > Subject: [Python.NET] application crash. > > > hello, > > the following application (python/C#) is working properly, when I do not > run simpy, a python framework. Once I enable, I get an immediate crash. > > See enclosed for runnable/compilable example. (run "python test.py"). To > run, it needs the Simpy framework, GPL, that you can get there: > http://simpy.sourceforge.net/ > > Simpy makes heavy uses of the "yield" construct. > > as soon as I uncomment that line in the script: > > #simulate() > > I get an immediate crash. I see no reason for this crash. > > I will open a tracker item. > > Stan. > > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From roman.yakovenko at gmail.com Thu Mar 31 07:51:40 2005 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Thu Mar 31 07:51:43 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available In-Reply-To: References: Message-ID: <7465b61705033021513896c595@mail.gmail.com> Hi. First of all thank you. Second - I still have problems to call static function from a few threads. Is it intentional? Do you want I send you test case? Roman On Mar 29, 2005 5:31 AM, Brian Lloyd wrote: > Python for .NET 1.0 beta 5 is now available - you can download > it from: > > http://www.zope.org/Members/Brian/PythonNet/ > > Python for .NET is a near-seamless integration of Python with the > .NET common language runtime. For more details, see the README: > > http://www.zope.org/Members/Brian/PythonNet/README.html > > The beta 5 release features a refactoring of thread management > support, making the runtime stable for multi-threaded applications, > as well as several improvements and bug-fixes. > > Special thanks to all who sent in thread-related scenarios and test > cases! I have tried to incorporate as many of these as possible into > the unit tests - if you still have problems that you suspect may be > related to threading or interpreter lock issues, please let me know > asap. > > There is also now a mailing list for discussion, questions and > issues related to Python for .NET at: pythondotnet@python.org. > > To subscribe to the mailing list or read the online archives, see: > > http://mail.python.org/mailman/listinfo/pythondotnet > > Brian Lloyd brian@zope.com > V.P. Engineering 540.361.1716 > Zope Corporation http://www.zope.com > > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From stan at phidani.be Wed Mar 30 18:52:24 2005 From: stan at phidani.be (Stan Pinte) Date: Thu Mar 31 09:37:11 2005 Subject: [Python.NET] Re: application crash. In-Reply-To: <424AD10C.3040500@phidani.be> References: <424AD10C.3040500@phidani.be> Message-ID: <424AD948.10006@phidani.be> Stan Pinte a ?crit : >hello, > >the following application (python/C#) is working properly, when I do not >run simpy, a python framework. Once I enable, I get an immediate crash. > > hello, I have skimmed the test case, and removed all dependancies toward the Simpy framework. Now it crashes once the python main loop is finished. If I comment out the code used to pass a python function as a delegate to the C# frame, it doesn't. here is the code: #test to show problem. import sys,time sys.path.append('.') from CLR.System.Reflection import Assembly a = Assembly.LoadWithPartialName("testpython") from CLR.testpython import MainForm, TestDelegate def test(): return "hop" if __name__ == '__main__': form = MainForm.GetInstance() form.TestDelegateProperty = TestDelegate(test) form.Start() time.sleep(1) print "done" Stan. >See enclosed for runnable/compilable example. (run "python test.py"). To >run, it needs the Simpy framework, GPL, that you can get there: >http://simpy.sourceforge.net/ > >Simpy makes heavy uses of the "yield" construct. > >as soon as I uncomment that line in the script: > >#simulate() > >I get an immediate crash. I see no reason for this crash. > >I will open a tracker item. > >Stan. > > > > From stan at phidani.be Thu Mar 31 09:44:53 2005 From: stan at phidani.be (Stan Pinte) Date: Thu Mar 31 09:44:54 2005 Subject: [Python.NET] application crash. In-Reply-To: References: Message-ID: <424BAA75.406@phidani.be> Brian Lloyd a ?crit : >Hi Stan - is this with the most recent (b5) release of >Python for .NET? If not, could you try with that and see >if you get the same result? > > It is with the B5, as stated in the bug report. thanks! Stan. From brian at zope.com Thu Mar 31 16:44:18 2005 From: brian at zope.com (Brian Lloyd) Date: Thu Mar 31 16:44:42 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available In-Reply-To: <7465b61705033021513896c595@mail.gmail.com> Message-ID: Yes, if you can send a test case that would help. Note that to be thread-safe and reentrancy-safe you also need to make sure that any managed code you have that uses Python.Runtime objects or methods is acquiring the Python lock first. For example: public class Example { public static int MethodThatCallsPython() { // have to acquire the GIL to safely call back into // Python!! IntPtr state = PythonEngine.AcquireLock(); PythonEngine.RunSimpleString("print 'hello'"); // done using python for now, so release the GIL... PythonEngine.ReleaseLock(state); } } Forgetting to manage the lock around a callback into Python is almost certain to cause a NullReferenceException (because there will be no valid thread state when the callback is made). Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com > -----Original Message----- > From: pythondotnet-bounces@python.org > [mailto:pythondotnet-bounces@python.org]On Behalf Of Roman Yakovenko > Sent: Thursday, March 31, 2005 12:52 AM > Cc: pythondotnet@python.org > Subject: Re: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available > > > Hi. First of all thank you. > > Second - I still have problems to call static function from a few > threads. Is it intentional? > Do you want I send you test case? > > Roman > > > On Mar 29, 2005 5:31 AM, Brian Lloyd wrote: > > Python for .NET 1.0 beta 5 is now available - you can download > > it from: > > > > http://www.zope.org/Members/Brian/PythonNet/ > > > > Python for .NET is a near-seamless integration of Python with the > > .NET common language runtime. For more details, see the README: > > > > http://www.zope.org/Members/Brian/PythonNet/README.html > > > > The beta 5 release features a refactoring of thread management > > support, making the runtime stable for multi-threaded applications, > > as well as several improvements and bug-fixes. > > > > Special thanks to all who sent in thread-related scenarios and test > > cases! I have tried to incorporate as many of these as possible into > > the unit tests - if you still have problems that you suspect may be > > related to threading or interpreter lock issues, please let me know > > asap. > > > > There is also now a mailing list for discussion, questions and > > issues related to Python for .NET at: pythondotnet@python.org. > > > > To subscribe to the mailing list or read the online archives, see: > > > > http://mail.python.org/mailman/listinfo/pythondotnet > > > > Brian Lloyd brian@zope.com > > V.P. Engineering 540.361.1716 > > Zope Corporation http://www.zope.com > > > > _________________________________________________ > > Python.NET mailing list - PythonDotNet@python.org > > http://mail.python.org/mailman/listinfo/pythondotnet > > > _________________________________________________ > Python.NET mailing list - PythonDotNet@python.org > http://mail.python.org/mailman/listinfo/pythondotnet > From stan at phidani.be Thu Mar 31 16:50:22 2005 From: stan at phidani.be (Stan Pinte) Date: Thu Mar 31 16:50:25 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available In-Reply-To: References: Message-ID: <424C0E2E.6020506@phidani.be> >Forgetting to manage the lock around a callback into Python >is almost certain to cause a NullReferenceException (because >there will be no valid thread state when the callback is >made). > > Is this take care of when a C# instance calls a delegate, wich is a python function? Or do we need to take the same care? (i.e. wrap the call to the delegate in a AcquireLock/ReleaseLock section). thanks! Stan. From pythondotnet at python.org Thu Mar 31 16:54:14 2005 From: pythondotnet at python.org (Collector: Python for .NET Issue ...) Date: Thu Mar 31 16:54:16 2005 Subject: [Python.NET] [PythonNet] 6/ 2 Resolve "Hang caused by multithreaded program that invokes .NET CLR" Message-ID: <20050331145635.A1351203365@mail.zope.org> Issue #6 Update (Resolve) "Hang caused by multithreaded program that invokes .NET CLR" Status Resolved, Threading/bug critical To followup, visit: http://www.zope.org/Members/Brian/PythonNet/Collector/6 ============================================================== = Resolve - Entry #2 by Brian on Mar 31, 2005 9:54 am Status: Pending => Resolved This issue is resolved in beta 5 - thanks for the test case (I put a condensed version of it into the unit tests). -BL ________________________________________ = Request - Entry #1 by Anonymous User on Mar 25, 2005 1:04 pm Hang caused by multithreaded program accessing CLR methods. I successfully reproduced the state with a very simple application. Here is very dumb python program that uses the CLR: from CLR.System import String class cSharpCaller( object ): def __init__( self ): print "Created a cSharpCaller instance" self.__cSharpString = None def createCSharpString( self, str ): self.__cSharpString = String( str ) def returnCSharpString( self ): return self.__cSharpString.ToString() And if I test this with this (dumb) threethreaded testprogram: import threading, thread, time from cSharpCaller import cSharpCaller def start_threads(amount=5): for i in range(amount): thread = threading.Thread(target=process_thread ) print "starting thread ", thread.getName() thread.start() def process_thread( ): print "thread started " for i in range( 2 ): print "Hi, I'm a thread" time.sleep( 1 ) cs = cSharpCaller( ) cs.createCSharpString( "created from a thread" ) print "cs.returnCSharpString() = ", cs.returnCSharpString() print "thread %s ended" start_threads( 3 ) I get this output: starting thread thread0 thread started Hi, I'm thread %s starting thread thread1 thread started Hi, I'm thread %s starting thread thread2 thread started Hi, I'm thread %s Hi, I'm thread %s Hi, I'm thread %s Hi, I'm thread %s Created a cSharpCaller instance Created a cSharpCaller instance cs.returnCSharpString() = and I have to CTRL+Pause to get out of the hang. Clearly hangs on the first String.ToString() method call. ============================================================== From brian at zope.com Thu Mar 31 16:55:29 2005 From: brian at zope.com (Brian Lloyd) Date: Thu Mar 31 16:55:51 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available In-Reply-To: <424C0E2E.6020506@phidani.be> Message-ID: > >Forgetting to manage the lock around a callback into Python > >is almost certain to cause a NullReferenceException (because > >there will be no valid thread state when the callback is > >made). > > > > > > Is this take care of when a C# instance calls a delegate, wich is a > python function? Or do we need to take the same care? (i.e. wrap the > call to the delegate in a AcquireLock/ReleaseLock section). > > thanks! > > Stan. No, I was just starting to look at your report, and I don't _think_ its the same thing. When PythonNet creates a delegate with a Python implementation, it generates the actual managed wrapper delegate and includes the necessary calls to AcquireLock() and ReleaseLock(). There are a bunch of unit tests for delegates that don't exhibit this behavior - the main difference I can see is that your delegate runs on a different thread (which shouldn't matter, but its a place to start). Brian Lloyd brian@zope.com V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com From stan at phidani.be Thu Mar 31 17:01:22 2005 From: stan at phidani.be (Stan Pinte) Date: Thu Mar 31 17:01:25 2005 Subject: [Python.NET] ANNOUNCE: Python for .NET beta 5 now available In-Reply-To: References: Message-ID: <424C10C2.4030400@phidani.be> Brian Lloyd a ?crit : >>>Forgetting to manage the lock around a callback into Python >>>is almost certain to cause a NullReferenceException (because >>>there will be no valid thread state when the callback is >>>made). >>> >>> >>> >>> >>Is this take care of when a C# instance calls a delegate, wich is a >>python function? Or do we need to take the same care? (i.e. wrap the >>call to the delegate in a AcquireLock/ReleaseLock section). >> >>thanks! >> >>Stan. >> >> > >No, I was just starting to look at your report, and I don't >_think_ its the same thing. > >When PythonNet creates a delegate with a Python implementation, >it generates the actual managed wrapper delegate and includes the >necessary calls to AcquireLock() and ReleaseLock(). > >There are a bunch of unit tests for delegates that don't exhibit >this behavior - the main difference I can see is that your >delegate runs on a different thread (which shouldn't matter, but >its a place to start). > > be sure to take a look at the comment I have made, because I include a second python test case, much simpler, without dependancies to Simpy. thanks a lot Bryan! >Brian Lloyd brian@zope.com >V.P. Engineering 540.361.1716 >Zope Corporation http://www.zope.com > > > > > > From pythondotnet at python.org Thu Mar 31 20:21:15 2005 From: pythondotnet at python.org (Collector: Python for .NET Issue ...) Date: Thu Mar 31 20:21:17 2005 Subject: [Python.NET] [PythonNet] 8/ 4 Resolve "Reentrancy bug" Message-ID: <20050331182336.DD6C12033B3@mail.zope.org> Issue #8 Update (Resolve) "Reentrancy bug" Status Resolved, General/bug medium To followup, visit: http://www.zope.org/Members/Brian/PythonNet/Collector/8 ============================================================== = Resolve - Entry #4 by Brian on Mar 31, 2005 1:21 pm Status: Pending => Resolved Great - thanks for the followup. I'm going to work on a 'tutorial for embedders' for the next release, as this isn't nearly well-documented enough... -BL ________________________________________ = Comment - Entry #3 by Anonymous User on Mar 31, 2005 12:59 pm You're right -- the bug is indeed caused by the lack of a lock acquisition in that code. Thanks for the tip. This item should be closed. > = Comment - Entry #2 by Brian on Mar 31, 2005 9:45 am > > Note that to be thread-safe and reentrancy-safe you > also need to make sure that any managed code you have > that uses Python.Runtime objects or methods is acquiring > the Python lock first. For example: > > public class Example { > > public static int MethodThatCallsPython() { > // have to acquire the GIL to safely call back into > // Python!! > IntPtr state = PythonEngine.AcquireLock(); > > PythonEngine.RunSimpleString("print 'hello'"); > > // done using python for now, so release the GIL... > PythonEngine.ReleaseLock(state); > } > > } > > Forgetting to manage the lock around a callback into Python > is almost certain to cause a NullReferenceException (because > there will be no valid thread state when the callback is > made). > > ________________________________________ > = Request - Entry #1 by Anonymous User on Mar 30, 2005 8:10 pm > > There is a reentrancy problem when calling CLR code from Python that > calls back into Python. The sequence I use to reproduce the problem is > fairly simple: > 1) Execute some Python code either through PyObject.InvokeMethod, or > PythonEngine.RunSimpleString. > 2) From the executed Python code, call into a CLR class. > 3) From the CLR class, execute Python code again. > > Here is some example code that demonstrates the issue: > using System; > using Python.Runtime; > > namespace PDNBug { > public class Example { > [STAThread] > static void Main(string[] args) { > PythonEngine.Initialize(); > PythonEngine.RunSimpleString("from CLR.PDNBug import > Example\nExample.FirstCall()"); > } > > public static void FirstCall () { > Console.WriteLine("First call succeeded."); > PythonEngine.RunSimpleString("from CLR.PDNBug import > Example\nExample.SecondCall()"); > } > > public static void SecondCall () { > Console.WriteLine("Second call succeeded."); > } > } > } > > The output generated is: > First call succeeded. > Traceback (most recent call last): > File "", line 2, in ? > System.NullReferenceException: Object reference not set to an instance of > an object. > at Python.Runtime.Runtime.PyRun_SimpleString(String code) > at Python.Runtime.PythonEngine.RunSimpleString(String code) > at PDNBug.Example.FirstCall() in example.cs:line 15: Object reference > not set to an instance of an object. > > The expected output is: > First call succeeded. > Second call succeeded. > > This issue was previously posted to the mailing list > (http://mail.python.org/pipermail/pythondotnet/2005-January/000235.html). ________________________________________ = Comment - Entry #2 by Brian on Mar 31, 2005 9:45 am Note that to be thread-safe and reentrancy-safe you also need to make sure that any managed code you have that uses Python.Runtime objects or methods is acquiring the Python lock first. For example: public class Example { public static int MethodThatCallsPython() { // have to acquire the GIL to safely call back into // Python!! IntPtr state = PythonEngine.AcquireLock(); PythonEngine.RunSimpleString("print 'hello'"); // done using python for now, so release the GIL... PythonEngine.ReleaseLock(state); } } Forgetting to manage the lock around a callback into Python is almost certain to cause a NullReferenceException (because there will be no valid thread state when the callback is made). ________________________________________ = Request - Entry #1 by Anonymous User on Mar 30, 2005 8:10 pm There is a reentrancy problem when calling CLR code from Python that calls back into Python. The sequence I use to reproduce the problem is fairly simple: 1) Execute some Python code either through PyObject.InvokeMethod, or PythonEngine.RunSimpleString. 2) From the executed Python code, call into a CLR class. 3) From the CLR class, execute Python code again. Here is some example code that demonstrates the issue: using System; using Python.Runtime; namespace PDNBug { public class Example { [STAThread] static void Main(string[] args) { PythonEngine.Initialize(); PythonEngine.RunSimpleString("from CLR.PDNBug import Example\nExample.FirstCall()"); } public static void FirstCall () { Console.WriteLine("First call succeeded."); PythonEngine.RunSimpleString("from CLR.PDNBug import Example\nExample.SecondCall()"); } public static void SecondCall () { Console.WriteLine("Second call succeeded."); } } } The output generated is: First call succeeded. Traceback (most recent call last): File "", line 2, in ? System.NullReferenceException: Object reference not set to an instance of an object. at Python.Runtime.Runtime.PyRun_SimpleString(String code) at Python.Runtime.PythonEngine.RunSimpleString(String code) at PDNBug.Example.FirstCall() in example.cs:line 15: Object reference not set to an instance of an object. The expected output is: First call succeeded. Second call succeeded. This issue was previously posted to the mailing list (http://mail.python.org/pipermail/pythondotnet/2005-January/000235.html). ==============================================================