[Python.NET] Added 64-bit support, delegates/events broken?

[John Burnett]

If anyone is looking at this delegates/events thing, I have a cargo-cult style fix that "seems to" work, sorta.  Gory details follow, and I'm not comfortable saying "it's fixed!" because I really don't know what's going on below.  I'd love it if someone could help:

CodeGenerator is currently a static class, and it's only used by the static DelegateManager.  In turn, DelegateManager is only used a couple times by EventObject and DelegateObject.

Since the crashes are occurring inside the static ctor of CodeGenerator, I made it a non-static type, along with DelegateManager.  (I gave DelegateManager a private instance of CodeGenerator, and gave Python.Runtime.PythonEngine a public instance of DelegateManager that's created in PythonEngine.Initialize.  Event/DelegateObject were then updated to use that public instance).  That all done, the ctor of CodeGenerator then ran fine on import, and no crashes occurred.

However, after that I'm getting a few "Decref(NULL)" prints during the test runs.  Each of them is happening inside a Python.Runtime.PythonException.Dispose() call.  I'm calling "Console.WriteLine(Runtime.PyObject_Repr(this.PyType));" in those Dispose calls for info, and the three exceptions are:

"TypeError : handler() takes exactly 6 arguments (3 given)"
"TypeError : handler() takes exactly 2 arguments (3 given)"
"TypeError : unbound method handler() must be called with GenericHandler instance as first argument (got EventTest instance instead)"

...and all seem to be coming from the test_delegate suite running under either cpython 2.5.1, or the wrapped console from python.net.  Not sure if those are expected?

Lastly, for a few more clues, I initially kept everything above static, and gave CodeGenerator a public static Init method that PythonEngine.InitExt called (along with all CodeGenerator methods, just for grins).  The Init method only ran once (a la "if(!init){init=true;...}").  The odd thing that then happened during test runs was this:

****
Load clr import hook
PythonEngine.InitExt running
PythonEngine.Initialize called
PythonEngine.Initialize running
CodeGenerator.Init called
CodeGenerator.Init running
...................................................................E............
................................................................................
.........................................CodeGenerator.Init called
CodeGenerator.Init running

Unhandled Exception: System.InvalidCastException: Unable to cast object of type 'System.Reflection.Module' to type 'System.Reflection.Emit.ModuleBuilder'.
****

...notice CodeGenerator.Init never completed running the first time, but the tests kept going.  Further debug prints show it was happening on the line "aBuilder = Thread.GetDomain().DefineDynamicAssembly(aname, aa);" in CodeGenerator's static ctor (same place the other crashes were happening).  I'm guessing there is an exception happening there, but it's getting eaten somewhere above.

That all said... help? Brian? Christian? :)

John


-----Original Message-----
From: pythondotnet-bounces+jburnett=blizzard.com at python.org [mailto:pythondotnet-bounces+jburnett=blizzard.com at python.org] On Behalf Of John Burnett
Sent: Monday, November 19, 2007 12:02 PM
To: pythondotnet at python.org
Subject: Re: [Python.NET] Added 64-bit support, delegates/events broken?

Re-sending to get it posted to the maillist (really don't know how these
work, or how the threading is determined under
http://mail.python.org/pipermail/pythondotnet/2007-November/thread.html
). 

----

Ok, I've submitted the patches.  Note that I'm purely Windows here using
the Microsoft .NET framework, so I'm not sure what the fallout is on
mono or other platforms.  The meat of the changes boils down to two
lines in the clrmodule file, but I gotta say it was fun getting back to
the IL:).

That said, here's the details for the two crashes I'm seeing:

My 32-bit machine is running XP Pro, and my 64-bit machine is running
Vista64 Business.  Both machines have VS2005 and VS2008beta2 installed,
so I have .NET frameworks 2.0, 3.0, and 3.5.

In runtests.py, the two offending scripts are 'test_event' and
'test_delegate' (lines 36 and 38).  I have to comment out one to see the
other run, since they both cause hard crashes in python.exe with an
unhandled exception.

In all cases below, I'm running the precompiled python 2.5.1 downloaded
from python.org.

For the 32-bit runs, I'm using a release build of clr.pyd that I
compiled under VS2005 from a download of pythonnet-2.0-alpha2.zip from
SourceForge.  For the 64-bit runs, I'm running a release build of
clr.pyd compiled under VS2005 based on the patches I just submitted.
The "only" difference between the 32-bit and 64-bit exception traces is
that the 64-bit version has an extra entry in the call stack (a call to
Python.Runtime.CodeGenerator.DefineType(String name, Type basetype)).

Any ideas, or need any additional details that I can provide? :)

Cheers,
John



**** 32-bit, test_event ****
Unhandled Exception: System.TypeInitializationException: The type
initializer for 'Python.Runtime.CodeGenerator' threw an exception. --->
System.InvalidCastException: Unable to cast object of type
'System.Reflection.Module' to type
'System.Reflection.Emit.ModuleBuilder'.
   at
System.Reflection.Emit.AssemblyBuilderData.GetInMemoryAssemblyModule()
   at System.AppDomain.InternalDefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access, String dir, Evidence evidence,
PermissionSet requiredPermissions, PermissionSet optionalPermissions,
PermissionSet refusedPermissions, StackCrawlMark& stackMark,
IEnumerable`1 unsafeAssemblyAttributes)
   at System.AppDomain.DefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access)
   at Python.Runtime.CodeGenerator..cctor()
   --- End of inner exception stack trace ---
   at Python.Runtime.DelegateManager.GetDispatcher(Type dtype)
   at Python.Runtime.DelegateManager.GetDelegate(Type dtype, IntPtr
callable)
   at Python.Runtime.EventObject.AddEventHandler(IntPtr target, IntPtr
handler)
   at Python.Runtime.EventBinding.nb_inplace_add(IntPtr ob, IntPtr arg)
****************************

**** 64-bit, test_event ****
Unhandled Exception: System.TypeInitializationException: The type
initializer for 'Python.Runtime.CodeGenerator' threw an exception. --->
System.InvalidCastException: Unable to cast object of type
'System.Reflection.Module' to type
'System.Reflection.Emit.ModuleBuilder'.
   at
System.Reflection.Emit.AssemblyBuilderData.GetInMemoryAssemblyModule()
   at System.AppDomain.InternalDefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access, String dir, Evidence evidence,
PermissionSet requiredPermissions, PermissionSet optionalPermissions,
PermissionSet refusedPermissions, StackCrawlMark& stackMark,
IEnumerable`1 unsafeAssemblyAttributes)
   at System.AppDomain.DefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access)
   at Python.Runtime.CodeGenerator..cctor()
   --- End of inner exception stack trace ---
   at Python.Runtime.CodeGenerator.DefineType(String name, Type
basetype)
   at Python.Runtime.DelegateManager.GetDispatcher(Type dtype)
   at Python.Runtime.DelegateManager.GetDelegate(Type dtype, IntPtr
callable)
   at Python.Runtime.EventObject.AddEventHandler(IntPtr target, IntPtr
handler)
   at Python.Runtime.EventBinding.nb_inplace_add(IntPtr ob, IntPtr arg)
****************************

**** 32-bit, test_delegate ****
Unhandled Exception: System.TypeInitializationException: The type
initializer for 'Python.Runtime.CodeGenerator' threw an exception. --->
System.InvalidCastException: Unable to cast object of type
'System.Reflection.Module' to type
'System.Reflection.Emit.ModuleBuilder'.
   at
System.Reflection.Emit.AssemblyBuilderData.GetInMemoryAssemblyModule()
   at System.AppDomain.InternalDefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access, String dir, Evidence evidence,
PermissionSet requiredPermissions, PermissionSet optionalPermissions,
PermissionSet refusedPermissions, StackCrawlMark& stackMark,
IEnumerable`1 unsafeAssemblyAttributes)
   at System.AppDomain.DefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access)
   at Python.Runtime.CodeGenerator..cctor()
   --- End of inner exception stack trace ---
   at Python.Runtime.DelegateManager.GetDispatcher(Type dtype)
   at Python.Runtime.DelegateManager.GetDelegate(Type dtype, IntPtr
callable)
   at Python.Runtime.DelegateObject.tp_new(IntPtr tp, IntPtr args,
IntPtr kw)
   at e__NativeCall.Call_3(IntPtr , IntPtr , IntPtr , IntPtr )
   at Python.Runtime.MetaType.tp_call(IntPtr tp, IntPtr args, IntPtr kw)
****************************

**** 64-bit, test_delgate ****
Unhandled Exception: System.TypeInitializationException: The type
initializer for 'Python.Runtime.CodeGenerator' threw an exception. --->
System.InvalidCastException: Unable to cast object of type
'System.Reflection.Module' to type
'System.Reflection.Emit.ModuleBuilder'.
   at
System.Reflection.Emit.AssemblyBuilderData.GetInMemoryAssemblyModule()
   at System.AppDomain.InternalDefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access, String dir, Evidence evidence,
PermissionSet requiredPermissions, PermissionSet optionalPermissions,
PermissionSet refusedPermissions, StackCrawlMark& stackMark,
IEnumerable`1 unsafeAssemblyAttributes)
   at System.AppDomain.DefineDynamicAssembly(AssemblyName name,
AssemblyBuilderAccess access)
   at Python.Runtime.CodeGenerator..cctor()
   --- End of inner exception stack trace ---
   at Python.Runtime.CodeGenerator.DefineType(String name, Type
basetype)
   at Python.Runtime.DelegateManager.GetDispatcher(Type dtype)
   at Python.Runtime.DelegateManager.GetDelegate(Type dtype, IntPtr
callable)
   at Python.Runtime.DelegateObject.tp_new(IntPtr tp, IntPtr args,
IntPtr kw)
   at e__NativeCall.Call_3(IntPtr , IntPtr , IntPtr , IntPtr )
   at Python.Runtime.MetaType.tp_call(IntPtr tp, IntPtr args, IntPtr kw)
****************************



-----Original Message-----
From: Christian Heimes [mailto:lists at cheimes.de] 
Sent: Sunday, November 18, 2007 5:38 PM
To: John Burnett
Cc: Brian Lloyd
Subject: Re: [Python.NET] Added 64-bit support, delegates/events broken?

John Burnett wrote:
> I've made a few changes necessary to make python.net 64-bit
compatible.  I can check in the changes on Monday (unless I need an
account set up first?)  I also fixed a broken test case wrt. exceptions.

Neither Brian nor me have a 64bit CPU. Although the code tries to be
64bit compatible neither of us is able to verify the code. Thanks for
your work! :)

You cannot check in the code yourself. Can you please make two patches,
one for the 64bit and one for the unit tests, and post them in the bug
and feature tracker at sf.net?

Brian: I still don't have developer and technician roles in the bug
tracker. Can you please raise my level for the tracker, too?

> That said, I'm getting unrelated hard crashes when running the test
cases for either events or delegates (i.e. the whole test harness
crashes out).  Is anyone else seeing this, or is it something peculiar
with my setup (it's happening in both 32 and 64 bit Vista, compiled
under VS2005).  I can post details on Monday if necessary...

I haven't seen hard crashes for a while. I'm using VS 2005 and Mono
1.2.4 for testing. Please post some details.

Christian

_________________________________________________
Python.NET mailing list - PythonDotNet at python.org
http://mail.python.org/mailman/listinfo/pythondotnet