From serge.weinstock at uk.bnpparibas.com Thu Oct 2 07:51:57 2014 From: serge.weinstock at uk.bnpparibas.com (Serge WEINSTOCK) Date: Thu, 2 Oct 2014 05:51:57 +0000 Subject: [Python.NET] Python.Net for Python3: bug when using an embedded interpreter In-Reply-To: References:

Message-ID: Thanks, it?s working fine. I?ve created a pull request for handling Python3 int conversion to int32 in 32 bits python interpreters. Could you have a look at it? Serge From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas.com at python.org [mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas.com at python.org] Sent: 30 September 2014 19:16 To: pythondotnet at python.org Subject: Re: [Python.NET] Python.Net for Python3: bug when using an embedded interpreter Should be fixed now. Please re-open if it's still a problem (https://github.com/renshawbay/pythonnet/issues/13). cheers, Tony On Tue, Sep 30, 2014 at 4:57 PM, Tony Roberts > wrote: Hi Serge, I see what you're saying. It would think that bit just needs to be moved until after we're sure python has been initialized so it works in both cases. I don't see any need to move it into a separate python module; having it compiled into one is cleaner IMHO. If you're not subclassing .NET classes in python and so don't need python methods/properties reflected to .net you won't need it anyway, so you could just remove that bit of code from your fork for now. I'll take a look, but if you get round to it before I do then please feel free to submit a pull request. I'll create an issue in github so it doesn't get forgotten. thanks, Tony On Tue, Sep 30, 2014 at 11:59 AM, Serge WEINSTOCK > wrote: Hi Tony, You recently made a change which prevents me using the library in an embedded Python interpreter. In pythonengine.cs, you ?inject? decorators with the following code: //================================================================= public static void Initialize() { ..... IntPtr globals = Runtime.PyEval_GetGlobals(); PyDict locals = new PyDict(); try { IntPtr builtins = Runtime.PyEval_GetBuiltins(); Runtime.PyDict_SetItemString(locals.Handle, "__builtins__", builtins); var assembly = Assembly.GetExecutingAssembly(); using (Stream stream = assembly.GetManifestResourceStream("Python.Runtime.resources.clr.py")) using (StreamReader reader = new StreamReader(stream)) { // add the contents of clr.py to the module string clr_py = reader.ReadToEnd(); PyObject result = RunString(clr_py, globals, locals.Handle); if (null == result) throw new PythonException(); result.Dispose(); } //================================================================= When running from a Python script, it?s fine because the Python interpreter has already been initialized and global variables have been defined. The issue is that when running from an embedded interpreter. As I must first initialize the Python engine using PythonEngine.Initialize(). So when we reach this piece of code, there is no script running, no ?Python context?, so ?PyEval_GetGlobals? returns null and ?RunString? fails. We could create an artificial global dictionary but it won?t be available to the rest of the code. I think it should be best to move this code into a Python module. If run from a script (PyEval_GetGlobals() != null), you could inject the decorators with a simple ?from decorators_pythonnet import *?. If run from an embedded interpreter, we could add a line for running the same code once the engine has been initialized. What do you think? Serge Weinstock ___________________________________________________________ This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is prohibited. Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional disclosures. _________________________________________________ Python.NET mailing list - PythonDotNet at python.org https://mail.python.org/mailman/listinfo/pythondotnet ___________________________________________________________ This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is prohibited. Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional disclosures. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.akhiyarov at gmail.com Mon Oct 6 15:45:13 2014 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Mon, 6 Oct 2014 08:45:13 -0500 Subject: [Python.NET] import Python module in C# - multiple instances, not global/static for the class Message-ID: Is there any way to import the same Python module as multiple instances, e.g. for each instance of C# class/object? For now it looks like the same module is imported for each instance of C# object (essentially a static Python object) when I do this: using (Py.GIL()) { dynamic syspy = Py.Import("sys"); syspy.path.append(ConfigDirectory); mymodulepy= Py.Import("mymodulepy"); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.akhiyarov at gmail.com Mon Oct 6 15:51:56 2014 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Mon, 6 Oct 2014 08:51:56 -0500 Subject: [Python.NET] How to embed properly using Python for .NET In-Reply-To: References:

Message-ID: It turned out that PTVS was stepping through the use-handled exceptions for module imports, so the problem was not here. The original problem is that when importing modules as .py* files, then it fails. The only way it works is when the module is setup as a package with corresponding __init__.py file(s) in the subfolder(s). Let me know if anyone else can reproduce this issue? On Mon, Aug 4, 2014 at 1:11 PM, Mika S wrote: > What dependencies are missing? > I find that having a basic python installation on the target machine is > necessary. If that's not the case then copy your python folder with the > modules and they eggs on the target machine and set the PYTHONPATH > environment var from c#. > > > On Sat, Aug 2, 2014 at 9:50 PM, Denis Akhiyarov > wrote: > >> When I try to use >> >> PythonEngine.ImportModule(mymodulename) >> >> some of the optional modules in dependencies are attempted to be loaded >> (not required for module use without embedding). This results in return >> null from this method because some of these optional dependencies are not >> required and hence not available. What is the proper method to use in this >> PythonNET API for loading user-written module which depends on multiple >> other modules? >> >> I tried importhook.cs from both versions: >> >> https://github.com/pythonnet/pythonnet >> >> https://github.com/renshawbay/pythonnet >> >> >> >> http://stackoverflow.com/questions/25101718/how-to-embed-properly-using-python-for-net >> >> _________________________________________________ >> Python.NET mailing list - PythonDotNet at python.org >> https://mail.python.org/mailman/listinfo/pythondotnet >> > > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.akhiyarov at gmail.com Mon Oct 6 15:53:50 2014 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Mon, 6 Oct 2014 08:53:50 -0500 Subject: [Python.NET] How to embed properly using Python for .NET In-Reply-To: References:

Message-ID: here is a related SO question: http://stackoverflow.com/questions/25101718 On Mon, Oct 6, 2014 at 8:51 AM, Denis Akhiyarov wrote: > It turned out that PTVS was stepping through the use-handled exceptions > for module imports, so the problem was not here. > > The original problem is that when importing modules as .py* files, then it > fails. The only way it works is when the module is setup as a package with > corresponding __init__.py file(s) in the subfolder(s). > > Let me know if anyone else can reproduce this issue? > > On Mon, Aug 4, 2014 at 1:11 PM, Mika S wrote: > >> What dependencies are missing? >> I find that having a basic python installation on the target machine is >> necessary. If that's not the case then copy your python folder with the >> modules and they eggs on the target machine and set the PYTHONPATH >> environment var from c#. >> >> >> On Sat, Aug 2, 2014 at 9:50 PM, Denis Akhiyarov < >> denis.akhiyarov at gmail.com> wrote: >> >>> When I try to use >>> >>> PythonEngine.ImportModule(mymodulename) >>> >>> some of the optional modules in dependencies are attempted to be loaded >>> (not required for module use without embedding). This results in return >>> null from this method because some of these optional dependencies are not >>> required and hence not available. What is the proper method to use in this >>> PythonNET API for loading user-written module which depends on multiple >>> other modules? >>> >>> I tried importhook.cs from both versions: >>> >>> https://github.com/pythonnet/pythonnet >>> >>> https://github.com/renshawbay/pythonnet >>> >>> >>> >>> http://stackoverflow.com/questions/25101718/how-to-embed-properly-using-python-for-net >>> >>> _________________________________________________ >>> Python.NET mailing list - PythonDotNet at python.org >>> https://mail.python.org/mailman/listinfo/pythondotnet >>> >> >> >> _________________________________________________ >> Python.NET mailing list - PythonDotNet at python.org >> https://mail.python.org/mailman/listinfo/pythondotnet >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.akhiyarov at gmail.com Tue Oct 7 17:45:19 2014 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Tue, 7 Oct 2014 10:45:19 -0500 Subject: [Python.NET] import Python module in C# - multiple instances, not global/static for the class In-Reply-To: References: Message-ID: After some thinking, the better solution is to wrap the python code into a class, so that each appropriate instance is called from C# side. On Mon, Oct 6, 2014 at 8:45 AM, Denis Akhiyarov wrote: > Is there any way to import the same Python module as multiple instances, > e.g. for each instance of C# class/object? For now it looks like the same > module is imported for each instance of C# object (essentially a static > Python object) when I do this: > > using (Py.GIL()) > { > dynamic syspy = Py.Import("sys"); > syspy.path.append(ConfigDirectory); > mymodulepy= Py.Import("mymodulepy"); > } > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zaripov at electronxray.com Fri Oct 17 12:41:56 2014 From: zaripov at electronxray.com (zaripov at electronxray.com) Date: Fri, 17 Oct 2014 10:41:56 +0000 Subject: [Python.NET] glibc invalid pointer when importing clr Message-ID: <09798747AB731042AE390DF7F282092BABD2623C@exch-2k10.elektron.spb.su> Hi! First, sorry for my English. I'm trying to use pythonnet on Debian Wheezy. I'm checkout svn trunk, when install mono-2, and other dependencies. Building clr.so done without errors. On the next step I'm put clr.so, Python.Runtime.dll and Python.Runtime.dll.config in /usr/local/lib/python2.7/dist-packages. But when I'm try to use this, I'm got such traceback: root at debian-salt:/usr/local/lib/python2.7/dist-packages# python Python 2.7.3 (default, Mar 14 2014, 11:57:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import clr *** glibc detected *** python: free(): invalid pointer: 0xb6653180 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41)[0xb7618c41] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8)[0xb761a4a8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb761d5ed] /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf)[0xb6e3f7af] ======= Memory map: ======== 00010000-00081000 rw-p 00000000 00:00 0 08048000-0828e000 r-xp 00000000 08:05 23939 /usr/bin/python2.7 0828e000-0828f000 r--p 00245000 08:05 23939 /usr/bin/python2.7 0828f000-082e4000 rw-p 00246000 08:05 23939 /usr/bin/python2.7 082e4000-082f1000 rw-p 00000000 00:00 0 09e7a000-09fec000 rw-p 00000000 00:00 0 [heap] b6500000-b6521000 rw-p 00000000 00:00 0 b6521000-b6600000 ---p 00000000 00:00 0 b6652000-b6693000 rw-p 00000000 00:00 0 b6693000-b66ae000 r--p 00000000 08:05 46606 /usr/local/lib/python2.7/dist-packages/Python.Runtime.dll b66ae000-b66b7000 ---p 00000000 00:00 0 b66b7000-b67b0000 rw-p 00000000 00:00 0 b67b0000-b67cf000 ---p 00000000 00:00 0 b67cf000-b67e0000 rwxp 00000000 00:00 0 b67e0000-b67ea000 r-xp 00000000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67ea000-b67eb000 r--p 00009000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67eb000-b67ec000 rw-p 0000a000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67ec000-b67f5000 r-xp 00000000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f5000-b67f6000 r--p 00008000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f6000-b67f7000 rw-p 00009000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f7000-b680a000 r-xp 00000000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680a000-b680b000 r--p 00012000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680b000-b680c000 rw-p 00013000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680c000-b680e000 rw-p 00000000 00:00 0 b6814000-b6818000 rw-p 00000000 00:00 0 b6818000-b681a000 r-xp 00000000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681a000-b681b000 r--p 00001000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681b000-b681c000 rw-p 00002000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681c000-b6a96000 r--p 00000000 08:05 41796 /usr/lib/mono/2.0/mscorlib.dll b6a96000-b6aa7000 rwxp 00000000 00:00 0 b6aa7000-b6ae4000 r-xp 00000000 08:01 33080 /lib/i386-linux-gnu/libpcre.so.3.13.1 b6ae4000-b6ae5000 rw-p 0003c000 08:01 33080 /lib/i386-linux-gnu/libpcre.so.3.13.1 b6ae5000-b6bc5000 r-xp 00000000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bc5000-b6bc9000 r--p 000e0000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bc9000-b6bca000 rw-p 000e4000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bca000-b6bd1000 rw-p 00000000 00:00 0 b6bd1000-b6ccc000 r-xp 00000000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6ccc000-b6ccd000 r--p 000fa000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6ccd000-b6cce000 rw-p 000fb000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6cce000-b6f39000 r-xp 00000000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f39000-b6f3a000 r--p 0026b000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f3a000-b6f8f000 rw-p 0026c000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f8f000-b6f9c000 rw-p 00000000 00:00 0 b6f9c000-b6fa3000 r-xp 00000000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa3000-b6fa4000 r--p 00006000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa4000-b6fa5000 rw-p 00007000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa5000-b72b6000 r-xp 00000000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72b6000-b72b9000 r--p 00310000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72b9000-b72be000 rw-p 00313000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72be000-b72dd000 rw-p 00000000 00:00 0 b72de000-b72e4000 r-xp 00000000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e4000-b72e5000 r--p 00005000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e5000-b72e6000 rw-p 00006000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e6000-b72e7000 rw-s 00000000 00:11 939641 /run/shm/mono.7877 b72e7000-b72eb000 rw-p 00000000 00:00 0 b72eb000-b7308000 r-xp 00000000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b7308000-b730a000 r--p 0001c000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b730a000-b730b000 rw-p 0001e000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b730b000-b7343000 r-xp 00000000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7343000-b7344000 r--p 00038000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7344000-b7347000 rw-p 00039000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7347000-b7349000 rw-p 00000000 00:00 0 b7349000-b734d000 r-xp 00000000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734d000-b734e000 r--p 00003000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734e000-b734f000 rw-p 00004000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734f000-b74c6000 r--p 00000000 08:05 6381 /usr/lib/locale/locale-archive b74c6000-b758b000 rw-p 00000000 00:00 0 b758b000-b75a7000 r-xp 00000000 08:01 32523 /lib/i386-linux-gnu/libgcc_s.so.1 b75a7000-b75a8000 rw-p 0001b000 08:01 32523 /lib/i386-linux-gnu/libgcc_s.so.1 b75a8000-b7705000 r-xp 00000000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7705000-b7707000 r--p 0015d000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7707000-b7708000 rw-p 0015f000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7708000-b770b000 rw-p 00000000 00:00 0 b770b000-b772f000 r-xp 00000000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b772f000-b7730000 r--p 00023000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b7730000-b7731000 rw-p 00024000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b7731000-b7748000 r-xp 00000000 08:01 32615 /lib/i386-linux-gnu/libz.so.1.2.7 b7748000-b7749000 r--p 00016000 08:01 32615 /lib/i386-linux-gnu/libz.so.1.2.7Stacktrace: at (wrapper managed-to-native) Python.Runtime.Runtime.Py_Initialize () <0xffffffff> at Python.Runtime.Runtime.Initialize () <0x00013> at Python.Runtime.PythonEngine.Initialize () <0x00047> at (wrapper runtime-invoke) object.runtime_invoke_void (object,intptr,intptr,intptr) <0xffffffff> Native stacktrace: /usr/lib/libmono-2.0.so.1(+0xb5b43) [0xb705ab43] [0xb777c410] [0xb777c422] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x51) [0xb75d2681] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x182) [0xb75d5ab2] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x66b55) [0xb760eb55] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41) [0xb7618c41] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8) [0xb761a4a8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d) [0xb761d5ed] /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf) [0xb6e3f7af] Debug info from gdb: ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= Mono 2.10.8.1-8 libglib2.0-dev 2.33.12+really2.32.4-5 Have any suggestions how to fix this? Help very welcome! Best regards, Renat Zaripov From tony at pyxll.com Fri Oct 17 18:30:24 2014 From: tony at pyxll.com (Tony Roberts) Date: Fri, 17 Oct 2014 17:30:24 +0100 Subject: [Python.NET] glibc invalid pointer when importing clr In-Reply-To: <09798747AB731042AE390DF7F282092BABD2623C@exch-2k10.elektron.spb.su> References: <09798747AB731042AE390DF7F282092BABD2623C@exch-2k10.elektron.spb.su> Message-ID: Hi, probably your python has not been compiled with --enable-shared, and so it ends up loading the python core twice (once in the executable, and again in the so referenced by the pythonnet module). This means that the one pythonnet is using doesn't get initialized correctly. You could try building python yourself with --enable-shared. Here are some instructions (there are loads of others, I just found this on google): http://du-alter-schwede.azurewebsites.net/index.php/2012/07/26/how-to-build-python-so-that-a-shared-library-libpythonx-y-so-is-produced/ Otherwise you could use the npython binary that builds as part of the pythonnet project. You might also want to try the latest version of the pythonnet code from the develop branch on github. It's got a Travis CI build so does get built and tested on linux (ubuntu not debian though) - although you should be able to get the svn code working too. https://github.com/pythonnet/pythonnet https://travis-ci.org/pythonnet/pythonnet/builds cheers, Tony On Fri, Oct 17, 2014 at 11:41 AM, wrote: > Hi! > > First, sorry for my English. I'm trying to use pythonnet on Debian Wheezy. > I'm checkout svn trunk, when install mono-2, and other dependencies. > Building clr.so done without errors. On the next step I'm put clr.so, > Python.Runtime.dll and Python.Runtime.dll.config in > /usr/local/lib/python2.7/dist-packages. > But when I'm try to use this, I'm got such traceback: > > root at debian-salt:/usr/local/lib/python2.7/dist-packages# python > Python 2.7.3 (default, Mar 14 2014, 11:57:14) > [GCC 4.7.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import clr > *** glibc detected *** python: free(): invalid pointer: 0xb6653180 *** > ======= Backtrace: ========= > /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41)[0xb7618c41] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8)[0xb761a4a8] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb761d5ed] > /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf)[0xb6e3f7af] > ======= Memory map: ======== > 00010000-00081000 rw-p 00000000 00:00 0 > 08048000-0828e000 r-xp 00000000 08:05 23939 /usr/bin/python2.7 > 0828e000-0828f000 r--p 00245000 08:05 23939 /usr/bin/python2.7 > 0828f000-082e4000 rw-p 00246000 08:05 23939 /usr/bin/python2.7 > 082e4000-082f1000 rw-p 00000000 00:00 0 > 09e7a000-09fec000 rw-p 00000000 00:00 0 [heap] > b6500000-b6521000 rw-p 00000000 00:00 0 > b6521000-b6600000 ---p 00000000 00:00 0 > b6652000-b6693000 rw-p 00000000 00:00 0 > b6693000-b66ae000 r--p 00000000 08:05 46606 > /usr/local/lib/python2.7/dist-packages/Python.Runtime.dll > b66ae000-b66b7000 ---p 00000000 00:00 0 > b66b7000-b67b0000 rw-p 00000000 00:00 0 > b67b0000-b67cf000 ---p 00000000 00:00 0 > b67cf000-b67e0000 rwxp 00000000 00:00 0 > b67e0000-b67ea000 r-xp 00000000 08:01 32537 > /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so > b67ea000-b67eb000 r--p 00009000 08:01 32537 > /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so > b67eb000-b67ec000 rw-p 0000a000 08:01 32537 > /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so > b67ec000-b67f5000 r-xp 00000000 08:01 32539 > /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so > b67f5000-b67f6000 r--p 00008000 08:01 32539 > /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so > b67f6000-b67f7000 rw-p 00009000 08:01 32539 > /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so > b67f7000-b680a000 r-xp 00000000 08:01 32533 > /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so > b680a000-b680b000 r--p 00012000 08:01 32533 > /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so > b680b000-b680c000 rw-p 00013000 08:01 32533 > /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so > b680c000-b680e000 rw-p 00000000 00:00 0 > b6814000-b6818000 rw-p 00000000 00:00 0 > b6818000-b681a000 r-xp 00000000 08:05 10456 > /usr/lib/i386-linux-gnu/gconv/UTF-16.so > b681a000-b681b000 r--p 00001000 08:05 10456 > /usr/lib/i386-linux-gnu/gconv/UTF-16.so > b681b000-b681c000 rw-p 00002000 08:05 10456 > /usr/lib/i386-linux-gnu/gconv/UTF-16.so > b681c000-b6a96000 r--p 00000000 08:05 41796 > /usr/lib/mono/2.0/mscorlib.dll > b6a96000-b6aa7000 rwxp 00000000 00:00 0 > b6aa7000-b6ae4000 r-xp 00000000 08:01 33080 > /lib/i386-linux-gnu/libpcre.so.3.13.1 > b6ae4000-b6ae5000 rw-p 0003c000 08:01 33080 > /lib/i386-linux-gnu/libpcre.so.3.13.1 > b6ae5000-b6bc5000 r-xp 00000000 08:05 4041 > /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 > b6bc5000-b6bc9000 r--p 000e0000 08:05 4041 > /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 > b6bc9000-b6bca000 rw-p 000e4000 08:05 4041 > /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 > b6bca000-b6bd1000 rw-p 00000000 00:00 0 > b6bd1000-b6ccc000 r-xp 00000000 08:01 33092 > /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 > b6ccc000-b6ccd000 r--p 000fa000 08:01 33092 > /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 > b6ccd000-b6cce000 rw-p 000fb000 08:01 33092 > /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 > b6cce000-b6f39000 r-xp 00000000 08:05 38605 > /usr/lib/libpython2.7.so.1.0 > b6f39000-b6f3a000 r--p 0026b000 08:05 38605 > /usr/lib/libpython2.7.so.1.0 > b6f3a000-b6f8f000 rw-p 0026c000 08:05 38605 > /usr/lib/libpython2.7.so.1.0 > b6f8f000-b6f9c000 rw-p 00000000 00:00 0 > b6f9c000-b6fa3000 r-xp 00000000 08:01 32543 > /lib/i386-linux-gnu/i686/cmov/librt-2.13.so > b6fa3000-b6fa4000 r--p 00006000 08:01 32543 > /lib/i386-linux-gnu/i686/cmov/librt-2.13.so > b6fa4000-b6fa5000 rw-p 00007000 08:01 32543 > /lib/i386-linux-gnu/i686/cmov/librt-2.13.so > b6fa5000-b72b6000 r-xp 00000000 08:05 41753 > /usr/lib/libmono-2.0.so.1.0.0 > b72b6000-b72b9000 r--p 00310000 08:05 41753 > /usr/lib/libmono-2.0.so.1.0.0 > b72b9000-b72be000 rw-p 00313000 08:05 41753 > /usr/lib/libmono-2.0.so.1.0.0 > b72be000-b72dd000 rw-p 00000000 00:00 0 > b72de000-b72e4000 r-xp 00000000 08:01 32535 > /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so > b72e4000-b72e5000 r--p 00005000 08:01 32535 > /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so > b72e5000-b72e6000 rw-p 00006000 08:01 32535 > /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so > b72e6000-b72e7000 rw-s 00000000 00:11 939641 /run/shm/mono.7877 > b72e7000-b72eb000 rw-p 00000000 00:00 0 > b72eb000-b7308000 r-xp 00000000 08:01 32559 > /lib/i386-linux-gnu/libtinfo.so.5.9 > b7308000-b730a000 r--p 0001c000 08:01 32559 > /lib/i386-linux-gnu/libtinfo.so.5.9 > b730a000-b730b000 rw-p 0001e000 08:01 32559 > /lib/i386-linux-gnu/libtinfo.so.5.9 > b730b000-b7343000 r-xp 00000000 08:01 32576 > /lib/i386-linux-gnu/libreadline.so.6.2 > b7343000-b7344000 r--p 00038000 08:01 32576 > /lib/i386-linux-gnu/libreadline.so.6.2 > b7344000-b7347000 rw-p 00039000 08:01 32576 > /lib/i386-linux-gnu/libreadline.so.6.2 > b7347000-b7349000 rw-p 00000000 00:00 0 > b7349000-b734d000 r-xp 00000000 08:05 24426 > /usr/lib/python2.7/lib-dynload/readline.so > b734d000-b734e000 r--p 00003000 08:05 24426 > /usr/lib/python2.7/lib-dynload/readline.so > b734e000-b734f000 rw-p 00004000 08:05 24426 > /usr/lib/python2.7/lib-dynload/readline.so > b734f000-b74c6000 r--p 00000000 08:05 6381 > /usr/lib/locale/locale-archive > b74c6000-b758b000 rw-p 00000000 00:00 0 > b758b000-b75a7000 r-xp 00000000 08:01 32523 > /lib/i386-linux-gnu/libgcc_s.so.1 > b75a7000-b75a8000 rw-p 0001b000 08:01 32523 > /lib/i386-linux-gnu/libgcc_s.so.1 > b75a8000-b7705000 r-xp 00000000 08:01 32527 > /lib/i386-linux-gnu/i686/cmov/libc-2.13.so > b7705000-b7707000 r--p 0015d000 08:01 32527 > /lib/i386-linux-gnu/i686/cmov/libc-2.13.so > b7707000-b7708000 rw-p 0015f000 08:01 32527 > /lib/i386-linux-gnu/i686/cmov/libc-2.13.so > b7708000-b770b000 rw-p 00000000 00:00 0 > b770b000-b772f000 r-xp 00000000 08:01 32531 > /lib/i386-linux-gnu/i686/cmov/libm-2.13.so > b772f000-b7730000 r--p 00023000 08:01 32531 > /lib/i386-linux-gnu/i686/cmov/libm-2.13.so > b7730000-b7731000 rw-p 00024000 08:01 32531 > /lib/i386-linux-gnu/i686/cmov/libm-2.13.so > b7731000-b7748000 r-xp 00000000 08:01 32615 > /lib/i386-linux-gnu/libz.so.1.2.7 > b7748000-b7749000 r--p 00016000 08:01 32615 > /lib/i386-linux-gnu/libz.so.1.2.7Stacktrace: > > at (wrapper managed-to-native) Python.Runtime.Runtime.Py_Initialize () > <0xffffffff> > at Python.Runtime.Runtime.Initialize () <0x00013> > at Python.Runtime.PythonEngine.Initialize () <0x00047> > at (wrapper runtime-invoke) object.runtime_invoke_void > (object,intptr,intptr,intptr) <0xffffffff> > > Native stacktrace: > > /usr/lib/libmono-2.0.so.1(+0xb5b43) [0xb705ab43] > [0xb777c410] > [0xb777c422] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x51) [0xb75d2681] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x182) [0xb75d5ab2] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x66b55) [0xb760eb55] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41) [0xb7618c41] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8) [0xb761a4a8] > /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d) [0xb761d5ed] > /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf) > [0xb6e3f7af] > > Debug info from gdb: > > > ================================================================= > Got a SIGABRT while executing native code. This usually indicates > a fatal error in the mono runtime or one of the native libraries > used by your application. > ================================================================= > > Mono 2.10.8.1-8 > libglib2.0-dev 2.33.12+really2.32.4-5 > > Have any suggestions how to fix this? Help very welcome! > > Best regards, > Renat Zaripov > > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zaripov at electronxray.com Sat Oct 18 22:54:47 2014 From: zaripov at electronxray.com (zaripov at electronxray.com) Date: Sat, 18 Oct 2014 20:54:47 +0000 Subject: [Python.NET] HA: glibc invalid pointer when importing clr In-Reply-To: References: <09798747AB731042AE390DF7F282092BABD2623C@exch-2k10.elektron.spb.su>, Message-ID: <09798747AB731042AE390DF7F282092BABD28316@exch-2k10.elektron.spb.su> Hi Tony, Your assumption about build option --enable-shared absolutely correct. Simple way to check this with using ldd: root at debian-salt:~# ldd /usr/bin/python2.7 linux-gate.so.1 (0xb772c000) libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xb7702000) libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0xb76fd000) libutil.so.1 => /lib/i386-linux-gnu/i686/cmov/libutil.so.1 (0xb76f8000) libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xb76db000) libm.so.6 => /lib/i386-linux-gnu/i686/cmov/libm.so.6 (0xb7695000) libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb74ea000) /lib/ld-linux.so.2 (0xb772d000) root at debian-salt:~# ldd /usr/local/lib/python2.7/dist-packages/pythonnet-2.0.0.dev1-py2.7-linux-i686.egg/EGG-INFO/scripts/npython linux-gate.so.1 (0xb7727000) libmonoboehm-2.0.so.1 => /usr/lib/libmonoboehm-2.0.so.1 (0xb7325000) libm.so.6 => /lib/i386-linux-gnu/i686/cmov/libm.so.6 (0xb72df000) librt.so.1 => /lib/i386-linux-gnu/i686/cmov/librt.so.1 (0xb72d5000) libdl.so.2 => /lib/i386-linux-gnu/i686/cmov/libdl.so.2 (0xb72d0000) libpthread.so.0 => /lib/i386-linux-gnu/i686/cmov/libpthread.so.0 (0xb72b4000) libglib-2.0.so.0 => /lib/i386-linux-gnu/libglib-2.0.so.0 (0xb718c000) libpython2.7.so.1.0 => /usr/lib/i386-linux-gnu/libpython2.7.so.1.0 (0xb6de5000) libc.so.6 => /lib/i386-linux-gnu/i686/cmov/libc.so.6 (0xb6c39000) /lib/ld-linux.so.2 (0xb7728000) libpcre.so.3 => /lib/i386-linux-gnu/libpcre.so.3 (0xb6bc8000) libz.so.1 => /lib/i386-linux-gnu/libz.so.1 (0xb6bab000) libutil.so.1 => /lib/i386-linux-gnu/i686/cmov/libutil.so.1 (0xb6ba7000) As you can see libpython2.7.so.1.0 not linked with python that shipped with debian. For me using npython is prefer, than build and replace python. Building pythonnet on Debian done well, with minor problems. Because Wheezy contains stable versions (in other words old) of packages, I'm using mono and gcc packages from sid. I'll thinking that pythonnet is dead when looked into sourceforge. How I'm not found this project on GitHub? It is strange... Thank you for explanation and advices, they helped me! Best regards, Renat Zaripov ________________________________ ??: PythonDotNet [pythondotnet-bounces+zaripov=electronxray.com at python.org] ?? ????? Tony Roberts [tony at pyxll.com] ??????????: 17 ??????? 2014 ?. 20:30 To: A list for users and developers of Python for .NET ????: Re: [Python.NET] glibc invalid pointer when importing clr Hi, probably your python has not been compiled with --enable-shared, and so it ends up loading the python core twice (once in the executable, and again in the so referenced by the pythonnet module). This means that the one pythonnet is using doesn't get initialized correctly. You could try building python yourself with --enable-shared. Here are some instructions (there are loads of others, I just found this on google): http://du-alter-schwede.azurewebsites.net/index.php/2012/07/26/how-to-build-python-so-that-a-shared-library-libpythonx-y-so-is-produced/ Otherwise you could use the npython binary that builds as part of the pythonnet project. You might also want to try the latest version of the pythonnet code from the develop branch on github. It's got a Travis CI build so does get built and tested on linux (ubuntu not debian though) - although you should be able to get the svn code working too. https://github.com/pythonnet/pythonnet https://travis-ci.org/pythonnet/pythonnet/builds cheers, Tony On Fri, Oct 17, 2014 at 11:41 AM, > wrote: Hi! First, sorry for my English. I'm trying to use pythonnet on Debian Wheezy. I'm checkout svn trunk, when install mono-2, and other dependencies. Building clr.so done without errors. On the next step I'm put clr.so, Python.Runtime.dll and Python.Runtime.dll.config in /usr/local/lib/python2.7/dist-packages. But when I'm try to use this, I'm got such traceback: root at debian-salt:/usr/local/lib/python2.7/dist-packages# python Python 2.7.3 (default, Mar 14 2014, 11:57:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import clr *** glibc detected *** python: free(): invalid pointer: 0xb6653180 *** ======= Backtrace: ========= /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41)[0xb7618c41] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8)[0xb761a4a8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d)[0xb761d5ed] /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf)[0xb6e3f7af] ======= Memory map: ======== 00010000-00081000 rw-p 00000000 00:00 0 08048000-0828e000 r-xp 00000000 08:05 23939 /usr/bin/python2.7 0828e000-0828f000 r--p 00245000 08:05 23939 /usr/bin/python2.7 0828f000-082e4000 rw-p 00246000 08:05 23939 /usr/bin/python2.7 082e4000-082f1000 rw-p 00000000 00:00 0 09e7a000-09fec000 rw-p 00000000 00:00 0 [heap] b6500000-b6521000 rw-p 00000000 00:00 0 b6521000-b6600000 ---p 00000000 00:00 0 b6652000-b6693000 rw-p 00000000 00:00 0 b6693000-b66ae000 r--p 00000000 08:05 46606 /usr/local/lib/python2.7/dist-packages/Python.Runtime.dll b66ae000-b66b7000 ---p 00000000 00:00 0 b66b7000-b67b0000 rw-p 00000000 00:00 0 b67b0000-b67cf000 ---p 00000000 00:00 0 b67cf000-b67e0000 rwxp 00000000 00:00 0 b67e0000-b67ea000 r-xp 00000000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67ea000-b67eb000 r--p 00009000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67eb000-b67ec000 rw-p 0000a000 08:01 32537 /lib/i386-linux-gnu/i686/cmov/libnss_files-2.13.so b67ec000-b67f5000 r-xp 00000000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f5000-b67f6000 r--p 00008000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f6000-b67f7000 rw-p 00009000 08:01 32539 /lib/i386-linux-gnu/i686/cmov/libnss_nis-2.13.so b67f7000-b680a000 r-xp 00000000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680a000-b680b000 r--p 00012000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680b000-b680c000 rw-p 00013000 08:01 32533 /lib/i386-linux-gnu/i686/cmov/libnsl-2.13.so b680c000-b680e000 rw-p 00000000 00:00 0 b6814000-b6818000 rw-p 00000000 00:00 0 b6818000-b681a000 r-xp 00000000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681a000-b681b000 r--p 00001000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681b000-b681c000 rw-p 00002000 08:05 10456 /usr/lib/i386-linux-gnu/gconv/UTF-16.so b681c000-b6a96000 r--p 00000000 08:05 41796 /usr/lib/mono/2.0/mscorlib.dll b6a96000-b6aa7000 rwxp 00000000 00:00 0 b6aa7000-b6ae4000 r-xp 00000000 08:01 33080 /lib/i386-linux-gnu/libpcre.so.3.13.1 b6ae4000-b6ae5000 rw-p 0003c000 08:01 33080 /lib/i386-linux-gnu/libpcre.so.3.13.1 b6ae5000-b6bc5000 r-xp 00000000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bc5000-b6bc9000 r--p 000e0000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bc9000-b6bca000 rw-p 000e4000 08:05 4041 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 b6bca000-b6bd1000 rw-p 00000000 00:00 0 b6bd1000-b6ccc000 r-xp 00000000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6ccc000-b6ccd000 r--p 000fa000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6ccd000-b6cce000 rw-p 000fb000 08:01 33092 /lib/i386-linux-gnu/libglib-2.0.so.0.3200.4 b6cce000-b6f39000 r-xp 00000000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f39000-b6f3a000 r--p 0026b000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f3a000-b6f8f000 rw-p 0026c000 08:05 38605 /usr/lib/libpython2.7.so.1.0 b6f8f000-b6f9c000 rw-p 00000000 00:00 0 b6f9c000-b6fa3000 r-xp 00000000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa3000-b6fa4000 r--p 00006000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa4000-b6fa5000 rw-p 00007000 08:01 32543 /lib/i386-linux-gnu/i686/cmov/librt-2.13.so b6fa5000-b72b6000 r-xp 00000000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72b6000-b72b9000 r--p 00310000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72b9000-b72be000 rw-p 00313000 08:05 41753 /usr/lib/libmono-2.0.so.1.0.0 b72be000-b72dd000 rw-p 00000000 00:00 0 b72de000-b72e4000 r-xp 00000000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e4000-b72e5000 r--p 00005000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e5000-b72e6000 rw-p 00006000 08:01 32535 /lib/i386-linux-gnu/i686/cmov/libnss_compat-2.13.so b72e6000-b72e7000 rw-s 00000000 00:11 939641 /run/shm/mono.7877 b72e7000-b72eb000 rw-p 00000000 00:00 0 b72eb000-b7308000 r-xp 00000000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b7308000-b730a000 r--p 0001c000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b730a000-b730b000 rw-p 0001e000 08:01 32559 /lib/i386-linux-gnu/libtinfo.so.5.9 b730b000-b7343000 r-xp 00000000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7343000-b7344000 r--p 00038000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7344000-b7347000 rw-p 00039000 08:01 32576 /lib/i386-linux-gnu/libreadline.so.6.2 b7347000-b7349000 rw-p 00000000 00:00 0 b7349000-b734d000 r-xp 00000000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734d000-b734e000 r--p 00003000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734e000-b734f000 rw-p 00004000 08:05 24426 /usr/lib/python2.7/lib-dynload/readline.so b734f000-b74c6000 r--p 00000000 08:05 6381 /usr/lib/locale/locale-archive b74c6000-b758b000 rw-p 00000000 00:00 0 b758b000-b75a7000 r-xp 00000000 08:01 32523 /lib/i386-linux-gnu/libgcc_s.so.1 b75a7000-b75a8000 rw-p 0001b000 08:01 32523 /lib/i386-linux-gnu/libgcc_s.so.1 b75a8000-b7705000 r-xp 00000000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7705000-b7707000 r--p 0015d000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7707000-b7708000 rw-p 0015f000 08:01 32527 /lib/i386-linux-gnu/i686/cmov/libc-2.13.so b7708000-b770b000 rw-p 00000000 00:00 0 b770b000-b772f000 r-xp 00000000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b772f000-b7730000 r--p 00023000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b7730000-b7731000 rw-p 00024000 08:01 32531 /lib/i386-linux-gnu/i686/cmov/libm-2.13.so b7731000-b7748000 r-xp 00000000 08:01 32615 /lib/i386-linux-gnu/libz.so.1.2.7 b7748000-b7749000 r--p 00016000 08:01 32615 /lib/i386-linux-gnu/libz.so.1.2.7Stacktrace: at (wrapper managed-to-native) Python.Runtime.Runtime.Py_Initialize () <0xffffffff> at Python.Runtime.Runtime.Initialize () <0x00013> at Python.Runtime.PythonEngine.Initialize () <0x00047> at (wrapper runtime-invoke) object.runtime_invoke_void (object,intptr,intptr,intptr) <0xffffffff> Native stacktrace: /usr/lib/libmono-2.0.so.1(+0xb5b43) [0xb705ab43] [0xb777c410] [0xb777c422] /lib/i386-linux-gnu/i686/cmov/libc.so.6(gsignal+0x51) [0xb75d2681] /lib/i386-linux-gnu/i686/cmov/libc.so.6(abort+0x182) [0xb75d5ab2] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x66b55) [0xb760eb55] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x70c41) [0xb7618c41] /lib/i386-linux-gnu/i686/cmov/libc.so.6(+0x724a8) [0xb761a4a8] /lib/i386-linux-gnu/i686/cmov/libc.so.6(cfree+0x6d) [0xb761d5ed] /usr/lib/libpython2.7.so.1.0(PyString_InternInPlace+0xaf) [0xb6e3f7af] Debug info from gdb: ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= Mono 2.10.8.1-8 libglib2.0-dev 2.33.12+really2.32.4-5 Have any suggestions how to fix this? Help very welcome! Best regards, Renat Zaripov _________________________________________________ Python.NET mailing list - PythonDotNet at python.org https://mail.python.org/mailman/listinfo/pythondotnet From boris.burdin at yahoo.fr Mon Oct 20 11:13:33 2014 From: boris.burdin at yahoo.fr (Boris Burdin) Date: Mon, 20 Oct 2014 10:13:33 +0100 Subject: [Python.NET] cx_freeze and python.net Message-ID: <1413796413.57257.YahooMailNeo@web173106.mail.ir2.yahoo.com> Hello everybody, I'm trying to build a python.net application using cx-freeze . Once the application is built, I put in my final folder the Python.Runtime.dll. but it stil doesn't seem to work . Actually, when I run the .exe file, nothing is happening. So, I wonder if there were something else to do, another dll to put in the folder, or any option to use with cx_freeze when building the application ? i would be very grateful if you could help me on this matter .. Thanks in advance, Boris -------------- next part -------------- An HTML attachment was scrubbed... URL: From zaripov at electronxray.com Thu Oct 23 09:35:04 2014 From: zaripov at electronxray.com (zaripov at electronxray.com) Date: Thu, 23 Oct 2014 07:35:04 +0000 Subject: [Python.NET] cx_freeze and python.net In-Reply-To: <1413796413.57257.YahooMailNeo@web173106.mail.ir2.yahoo.com> References: <1413796413.57257.YahooMailNeo@web173106.mail.ir2.yahoo.com> Message-ID: <09798747AB731042AE390DF7F282092BABD45B9F@exch-2k10.elektron.spb.su> Hello Boris, I?m not worked with cx-freeze, but I use PyInstaller, that contains hook for clr library. This hook been broken in version 2.1, but this bug is simply fixed (see ticket http://www.pyinstaller.org/ticket/878 and patch here http://www.pyinstaller.org/changeset/04eb1cf43a3dfecf8825a4a476e8639c31960a7e/project ). I'm builded *.exe file with clr and Python.Runtime and if they placed in same directory, all work fine. I think build the one file bundle, that contains all needed libraries and dll?s also possible. Best regards, Renat Zaripov From: PythonDotNet [mailto:pythondotnet-bounces+zaripov=electronxray.com at python.org] On Behalf Of Boris Burdin Sent: Monday, October 20, 2014 1:14 PM To: pythondotnet at python.org; Boris Moi Subject: [Python.NET] cx_freeze and python.net Hello everybody, I'm trying to build a python.net application using cx-freeze . Once the application is built, I put in my final folder the Python.Runtime.dll. but it stil doesn't seem to work . Actually, when I run the .exe file, nothing is happening. So, I wonder if there were something else to do, another dll to put in the folder, or any option to use with cx_freeze when building the application ? i would be very grateful if you could help me on this matter .. Thanks in advance, Boris -------------- next part -------------- An HTML attachment was scrubbed... URL: From aklein at bluemountaincapital.com Thu Oct 23 16:55:29 2014 From: aklein at bluemountaincapital.com (Adam Klein) Date: Thu, 23 Oct 2014 14:55:29 +0000 Subject: [Python.NET] binding redirects Message-ID: <60CDCA83B420BB48BF95B8AF66CE7FFF45A147E8@NYMBX02.bcna.corp> Hi all, I recently had a situation where I wanted to redirect FSharp.Core.dll from 4.3.0.0 to 4.3.1.0 during Python.NET assembly loading, and the only way I could figure out how to do so was machine-wide via the machine.config in the .NET framework. I looked at the Python.NET source code and I'm fairly confident you can't do this at the "application" level right now, but I just wanted to ask anyway. Would be a nice feature to have, not sure it's possible with the current Load/LoadFrom assembly loading semantics? Best, Adam -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikhilgarg.gju at gmail.com Sun Oct 26 10:18:40 2014 From: nikhilgarg.gju at gmail.com (Nikhil Garg) Date: Sun, 26 Oct 2014 17:18:40 +0800 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array Message-ID: Hi, I have looked at the following thread to copy a c# array to numpy list using python dot net. https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html When I try the methods mentioned in the complete thread I manage to get the array from simple iteration method or np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work for my case. To explain about my issue, I use a computer code written in c# which produces output in a custom build format. I am able to access the data from the file system using python dot net, but it seems that it take almost half an hour to get the data from the file. For example my file has an array of size 5018x73x400, if i use *fromiter, *i provide the last bit of array to copy to python due to the way file stores output. for j in xrange(num_nodes-1): for i in xrange(nsteps): temp = file_in.ReadItemTimeStep(j+1,i).Data temp_1 = np.fromiter(temp, float) dest[j, i] = temp_1 Other way of doing same task would be to use a for loop but tht would be really slow. for k in xrange(num_nodes-1): for k in xrange(nsteps): for i in xrange(nvals): temp[k,j,i] = file_in.ReadItemTimeStep(k+1, j).Data[i] I am trying to find a faster way to extract the data from the file. I was wondering if there is a possibility to use something like GChandle or Marshal.Copy. I have also attached same output from Marshal.Copy Marshal.Copy(temp, 0, IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp)) 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 3.13551399e-242, 4.20993645e-161, 8.71245319e-119, 8.38370432e-089, 2.60762781e-075, 1.92616374e-072, 2.89006184e-072, 3.06007265e-082, 3.81879776e-315, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.03959661e-297, 7.22038448e-196, 3.05237954e-130, 2.58469346e-093, 2.29495911e-070, 3.74354820e-062, 1.74288060e-061, 2.13172553e-067, 2.74305386e-077, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.02484756e-240, 1.70193770e-154, 1.30625320e-108, 1.16793762e-075, 2.96889340e-061, 7.84687103e-058, 1.16879604e-057, 5.56256478e-068, 4.06437857e-315, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 1.11705239e-296, 1.46978156e-194, 1.98034372e-128, 3.79615170e-091, 5.88936509e-068, 1.85305929e-059, 2.16186001e-058, Whereas the actual values stored in the array are 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 5.66641200e-37, 1.24961600e-30, 5.34846800e-25, 1.72519600e-20, 1.49402200e-17, 3.27685300e-15, 4.10687500e-13, 1.87978200e-11, 2.01397900e-10, 8.83561100e-10, 2.14397600e-09, 2.02600300e-09, 2.47599800e-09, 2.15987100e-09, 4.82694100e-10, 1.18759300e-10, 3.32024100e-11, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.45288600e-37, 7.37245900e-31, 7.59282500e-25, 5.85463400e-20, 1.20634700e-16, 3.06414300e-14, 5.16041300e-12, 2.87562800e-10, 3.62841500e-09, 1.63540600e-08, 4.06988300e-08, 3.97764200e-08, 4.88768500e-08, 4.07385000e-08, 8.78868600e-09, I would appreciate any help i can get regarding this matter. Nikhil -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad at fie.us Mon Oct 27 17:50:35 2014 From: brad at fie.us (Bradley Friedman) Date: Mon, 27 Oct 2014 12:50:35 -0400 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array In-Reply-To: References: Message-ID: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us> You indicate that you are reading from a file. The thread you reference was about copying data in memory. I?d think matters of buffering and read-ahead caches would be far more relevant than anything else. Am I missing something? > On Oct 26, 2014, at 5:18 AM, Nikhil Garg wrote: > > > Hi, > > I have looked at the following thread to copy a c# array to numpy list using python dot net. > > https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html > > When I try the methods mentioned in the complete thread I manage to get the array from simple iteration method or > np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work for my case. > > To explain about my issue, I use a computer code written in c# which produces output in a custom build format. I am able to access the data from the file system using python dot net, but it seems that it take almost half an hour to get the data from the file. > > For example my file has an array of size 5018x73x400, if i use fromiter, i provide the last bit of array to copy to python due to the way file stores output. > > for j in xrange(num_nodes-1): > for i in xrange(nsteps): > temp = file_in.ReadItemTimeStep(j+1,i).Data > temp_1 = np.fromiter(temp, float) > dest[j, i] = temp_1 > > Other way of doing same task would be to use a for loop but tht would be really slow. > > for k in xrange(num_nodes-1): > for k in xrange(nsteps): > for i in xrange(nvals): > temp[k,j,i] = file_in.ReadItemTimeStep(k+1, j).Data[i] > > I am trying to find a faster way to extract the data from the file. I was wondering if there is a possibility to use something like GChandle or Marshal.Copy. I have also attached same output from Marshal.Copy > > Marshal.Copy(temp, 0, IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp)) > > > 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, > 0.00000000e+000, 3.13551399e-242, 4.20993645e-161, > 8.71245319e-119, 8.38370432e-089, 2.60762781e-075, > 1.92616374e-072, 2.89006184e-072, 3.06007265e-082, > 3.81879776e-315, 0.00000000e+000, 0.00000000e+000, > 0.00000000e+000, 1.03959661e-297, 7.22038448e-196, > 3.05237954e-130, 2.58469346e-093, 2.29495911e-070, > 3.74354820e-062, 1.74288060e-061, 2.13172553e-067, > 2.74305386e-077, 0.00000000e+000, 0.00000000e+000, > 0.00000000e+000, 0.00000000e+000, 1.02484756e-240, > 1.70193770e-154, 1.30625320e-108, 1.16793762e-075, > 2.96889340e-061, 7.84687103e-058, 1.16879604e-057, > 5.56256478e-068, 4.06437857e-315, 0.00000000e+000, > 0.00000000e+000, 0.00000000e+000, 1.11705239e-296, > 1.46978156e-194, 1.98034372e-128, 3.79615170e-091, > 5.88936509e-068, 1.85305929e-059, 2.16186001e-058, > > Whereas the actual values stored in the array are > 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, > 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, > 0.00000000e+00, 0.00000000e+00, 5.66641200e-37, > 1.24961600e-30, 5.34846800e-25, 1.72519600e-20, > 1.49402200e-17, 3.27685300e-15, 4.10687500e-13, > 1.87978200e-11, 2.01397900e-10, 8.83561100e-10, > 2.14397600e-09, 2.02600300e-09, 2.47599800e-09, > 2.15987100e-09, 4.82694100e-10, 1.18759300e-10, > 3.32024100e-11, 0.00000000e+00, 0.00000000e+00, > 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, > 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, > 1.45288600e-37, 7.37245900e-31, 7.59282500e-25, > 5.85463400e-20, 1.20634700e-16, 3.06414300e-14, > 5.16041300e-12, 2.87562800e-10, 3.62841500e-09, > 1.63540600e-08, 4.06988300e-08, 3.97764200e-08, > 4.88768500e-08, 4.07385000e-08, 8.78868600e-09, > > I would appreciate any help i can get regarding this matter. > > Nikhil > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikhilgarg.gju at gmail.com Tue Oct 28 14:55:13 2014 From: nikhilgarg.gju at gmail.com (Nikhil) Date: Tue, 28 Oct 2014 21:55:13 +0800 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array In-Reply-To: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us> References: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us> Message-ID: Hello, Yeah, I read data from a file say at each node and each time step, but when i try to use Marshal approach i get gibberish but when i use simple iter i get correct values. i have been trying the approach used in example in the previous post and that example makes sense but it doesnt make sense when i use it in my case. I am right now assigning it to a variable, i am now thinking of exploring the possibility of saving data to a dot net array maybe using System.Array and saving data to it but not sure if that even make sense. Sent from my iPhone > On 28 Oct 2014, at 12:50 am, Bradley Friedman wrote: > > You indicate that you are reading from a file. The thread you reference was about copying data in memory. I?d think matters of buffering and read-ahead caches would be far more relevant than anything else. Am I missing something? > >> On Oct 26, 2014, at 5:18 AM, Nikhil Garg wrote: >> >> >> Hi, >> >> I have looked at the following thread to copy a c# array to numpy list using python dot net. >> >> https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html >> >> When I try the methods mentioned in the complete thread I manage to get the array from simple iteration method or >> np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work for my case. >> >> To explain about my issue, I use a computer code written in c# which produces output in a custom build format. I am able to access the data from the file system using python dot net, but it seems that it take almost half an hour to get the data from the file. >> >> For example my file has an array of size 5018x73x400, if i use fromiter, i provide the last bit of array to copy to python due to the way file stores output. >> >> for j in xrange(num_nodes-1): >> for i in xrange(nsteps): >> temp = file_in.ReadItemTimeStep(j+1,i).Data >> temp_1 = np.fromiter(temp, float) >> dest[j, i] = temp_1 >> >> Other way of doing same task would be to use a for loop but tht would be really slow. >> >> for k in xrange(num_nodes-1): >> for k in xrange(nsteps): >> for i in xrange(nvals): >> temp[k,j,i] = file_in.ReadItemTimeStep(k+1, j).Data[i] >> >> I am trying to find a faster way to extract the data from the file. I was wondering if there is a possibility to use something like GChandle or Marshal.Copy. I have also attached same output from Marshal.Copy >> >> Marshal.Copy(temp, 0, IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp)) >> >> >> 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, >> 0.00000000e+000, 3.13551399e-242, 4.20993645e-161, >> 8.71245319e-119, 8.38370432e-089, 2.60762781e-075, >> 1.92616374e-072, 2.89006184e-072, 3.06007265e-082, >> 3.81879776e-315, 0.00000000e+000, 0.00000000e+000, >> 0.00000000e+000, 1.03959661e-297, 7.22038448e-196, >> 3.05237954e-130, 2.58469346e-093, 2.29495911e-070, >> 3.74354820e-062, 1.74288060e-061, 2.13172553e-067, >> 2.74305386e-077, 0.00000000e+000, 0.00000000e+000, >> 0.00000000e+000, 0.00000000e+000, 1.02484756e-240, >> 1.70193770e-154, 1.30625320e-108, 1.16793762e-075, >> 2.96889340e-061, 7.84687103e-058, 1.16879604e-057, >> 5.56256478e-068, 4.06437857e-315, 0.00000000e+000, >> 0.00000000e+000, 0.00000000e+000, 1.11705239e-296, >> 1.46978156e-194, 1.98034372e-128, 3.79615170e-091, >> 5.88936509e-068, 1.85305929e-059, 2.16186001e-058, >> >> Whereas the actual values stored in the array are >> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, >> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, >> 0.00000000e+00, 0.00000000e+00, 5.66641200e-37, >> 1.24961600e-30, 5.34846800e-25, 1.72519600e-20, >> 1.49402200e-17, 3.27685300e-15, 4.10687500e-13, >> 1.87978200e-11, 2.01397900e-10, 8.83561100e-10, >> 2.14397600e-09, 2.02600300e-09, 2.47599800e-09, >> 2.15987100e-09, 4.82694100e-10, 1.18759300e-10, >> 3.32024100e-11, 0.00000000e+00, 0.00000000e+00, >> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, >> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, >> 1.45288600e-37, 7.37245900e-31, 7.59282500e-25, >> 5.85463400e-20, 1.20634700e-16, 3.06414300e-14, >> 5.16041300e-12, 2.87562800e-10, 3.62841500e-09, >> 1.63540600e-08, 4.06988300e-08, 3.97764200e-08, >> 4.88768500e-08, 4.07385000e-08, 8.78868600e-09, >> >> I would appreciate any help i can get regarding this matter. >> >> Nikhil >> _________________________________________________ >> Python.NET mailing list - PythonDotNet at python.org >> https://mail.python.org/mailman/listinfo/pythondotnet > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet -------------- next part -------------- An HTML attachment was scrubbed... URL: From brad at fie.us Tue Oct 28 22:45:14 2014 From: brad at fie.us (Bradley Friedman) Date: Tue, 28 Oct 2014 17:45:14 -0400 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array In-Reply-To: References: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us> Message-ID: Well it makes sense to me that doing it via an iterator, and element at a time, would be slow. There?s a lot of call overhead associated with each iteration step. Whether it?s done in .net, or in python, or a call from one to the other, it will be slow. It?s still a call where you?d be better off copying whole buffers. Ideally you?d pull the data into as simple and raw a data structure as you can on the dotnet side, in a buffered manner. Then you?d execute a movement of the data across, a reasonably sized chunk of buffer at a time. This will reduce call overhead and also allow read-ahead caching to do its thing on the file-access side of things. Your suggestion of loading into a .net array and then moving that array over, makes sense. But I think it comes down to what you can do with the third party file-format library. If its not going to provide you with the data as some kind of buffer with a cohesive and known format in memory, you?re not really going to be able to move it over without iterating over it and reformatting it at some point. Specifically, I?d point to Jeffery?s original caveat: "but does involve a number of assumptions (for example that the data in the two arrays are laid out in the same way)." The question is: is there a way to get the data off of disk and in memory from dotnet library, where its layout in memory is known, and something you want exactly as it is, but in python? If so, you should be able to use the methods from the afore linked thread. If not, you?re probably stuck iterating somewhere to reformat it, no matter what. Which is probably why you got garbage back. I?m guessing the object returned from the dotnet file-format-library isn?t laid out right, as suggested in the afore referenced caveat. > On Oct 28, 2014, at 9:55 AM, Nikhil wrote: > > Hello, > Yeah, I read data from a file say at each node and each time step, but when i try to use Marshal approach i get gibberish but when i use simple iter i get correct values. i have been trying the approach used in example in the previous post and that example makes sense but it doesnt make sense when i use it in my case. I am right now assigning it to a variable, i am now thinking of exploring the possibility of saving data to a dot net array maybe using System.Array and saving data to it but not sure if that even make sense. > > Sent from my iPhone From jeff at coderforlife.com Wed Oct 29 04:04:32 2014 From: jeff at coderforlife.com (Jeffrey Bush) Date: Tue, 28 Oct 2014 20:04:32 -0700 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array In-Reply-To: References: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us>

Message-ID: I finally have a chance to chime in, and Bradley is exactly right. Marshall.Copy copies the raw data, and apparently your file library does not store that data in a nice, contiguous, manner. While it is highly likely that copying all the data to an array in C# will be faster than the fromiter in Python, I am unsure if copying all the data to an array in C# then copying all the data again to a numpy array will be faster than fromiter (cause you have to copy it twice). The exception is if the file library has a function like ToArray that is optimized to copy the data to a linear chunk of data. So, what type is "Data"? Another factor is how long the chunk of data you are copying is. You say the last axis is only 400 elements long. Check out my code and you will see that at 400 elements long, fromiter is actually the fastest (at least when I tried). An example run: Copy using for loop in 0.000884 sec Copy using fromiter in 0.000144 sec # fastest Copy using fromstring in 0.001460 sec # fairly slow, 10.3x slower than fromiter Copy using Marshal.Copy in 0.001680 sec # slowest, 11.7x slower than fromiter I start to do better with Marshal.Copy then fromiter around 5000 elements copied. This is because the overhead of the mass copies is high but adding each element doesn't take much time. fromstring has a lower overhead but slightly longer per-element time (fromstring is better than Marshal.Copy until ~200,000 elements). So you might be doing as good as you can possibly do. If I knew more about your file format library I might be able to provide more insight. Jeff On Tue, Oct 28, 2014 at 2:45 PM, Bradley Friedman wrote: > Well it makes sense to me that doing it via an iterator, and element at a > time, would be slow. There?s a lot of call overhead associated with each > iteration step. Whether it?s done in .net, or in python, or a call from > one to the other, it will be slow. It?s still a call where you?d be better > off copying whole buffers. > > Ideally you?d pull the data into as simple and raw a data structure as you > can on the dotnet side, in a buffered manner. Then you?d execute a > movement of the data across, a reasonably sized chunk of buffer at a time. > This will reduce call overhead and also allow read-ahead caching to do its > thing on the file-access side of things. > > Your suggestion of loading into a .net array and then moving that array > over, makes sense. But I think it comes down to what you can do with the > third party file-format library. If its not going to provide you with the > data as some kind of buffer with a cohesive and known format in memory, > you?re not really going to be able to move it over without iterating over > it and reformatting it at some point. > > Specifically, I?d point to Jeffery?s original caveat: > > "but does involve a number of assumptions (for example that the data in > the two arrays are laid out in the same way)." > > The question is: is there a way to get the data off of disk and in memory > from dotnet library, where its layout in memory is known, and something you > want exactly as it is, but in python? If so, you should be able to use the > methods from the afore linked thread. If not, you?re probably stuck > iterating somewhere to reformat it, no matter what. Which is probably why > you got garbage back. I?m guessing the object returned from the dotnet > file-format-library isn?t laid out right, as suggested in the afore > referenced caveat. > > > > On Oct 28, 2014, at 9:55 AM, Nikhil wrote: > > > > Hello, > > Yeah, I read data from a file say at each node and each time step, but > when i try to use Marshal approach i get gibberish but when i use simple > iter i get correct values. i have been trying the approach used in example > in the previous post and that example makes sense but it doesnt make sense > when i use it in my case. I am right now assigning it to a variable, i am > now thinking of exploring the possibility of saving data to a dot net array > maybe using System.Array and saving data to it but not sure if that even > make sense. > > > > Sent from my iPhone > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nikhilgarg.gju at gmail.com Thu Oct 30 18:19:15 2014 From: nikhilgarg.gju at gmail.com (Nikhil Garg) Date: Fri, 31 Oct 2014 01:19:15 +0800 Subject: [Python.NET] Efficient copy of .NET Array to ctypes or numpy array In-Reply-To: References: <8BE6014E-0EB7-43B7-AF98-EB35C292E0D1@fie.us>

Message-ID: Thanks Brad and Jeff for the detailed info. For now, fromiter is serving me well and has reduced my processing time considerably, so I am just going to stick with it. On 29 October 2014 11:04, Jeffrey Bush wrote: > I finally have a chance to chime in, and Bradley is exactly right. > Marshall.Copy copies the raw data, and apparently your file library does > not store that data in a nice, contiguous, manner. While it is highly > likely that copying all the data to an array in C# will be faster than the > fromiter in Python, I am unsure if copying all the data to an array in C# > then copying all the data again to a numpy array will be faster than > fromiter (cause you have to copy it twice). The exception is if the file > library has a function like ToArray that is optimized to copy the data to a > linear chunk of data. So, what type is "Data"? > > Another factor is how long the chunk of data you are copying is. You say > the last axis is only 400 elements long. Check out my code and you will see > that at 400 elements long, fromiter is actually the fastest (at least when > I tried). An example run: > > Copy using for loop in 0.000884 sec > Copy using fromiter in 0.000144 sec # fastest > Copy using fromstring in 0.001460 sec # fairly slow, 10.3x slower than > fromiter > Copy using Marshal.Copy in 0.001680 sec # slowest, 11.7x slower than > fromiter > > I start to do better with Marshal.Copy then fromiter around 5000 elements > copied. This is because the overhead of the mass copies is high but adding > each element doesn't take much time. fromstring has a lower overhead but > slightly longer per-element time (fromstring is better than Marshal.Copy > until ~200,000 elements). > > So you might be doing as good as you can possibly do. If I knew more about > your file format library I might be able to provide more insight. > > Jeff > > On Tue, Oct 28, 2014 at 2:45 PM, Bradley Friedman wrote: > >> Well it makes sense to me that doing it via an iterator, and element at a >> time, would be slow. There?s a lot of call overhead associated with each >> iteration step. Whether it?s done in .net, or in python, or a call from >> one to the other, it will be slow. It?s still a call where you?d be better >> off copying whole buffers. >> >> Ideally you?d pull the data into as simple and raw a data structure as >> you can on the dotnet side, in a buffered manner. Then you?d execute a >> movement of the data across, a reasonably sized chunk of buffer at a time. >> This will reduce call overhead and also allow read-ahead caching to do its >> thing on the file-access side of things. >> >> Your suggestion of loading into a .net array and then moving that array >> over, makes sense. But I think it comes down to what you can do with the >> third party file-format library. If its not going to provide you with the >> data as some kind of buffer with a cohesive and known format in memory, >> you?re not really going to be able to move it over without iterating over >> it and reformatting it at some point. >> >> Specifically, I?d point to Jeffery?s original caveat: >> >> "but does involve a number of assumptions (for example that the data in >> the two arrays are laid out in the same way)." >> >> The question is: is there a way to get the data off of disk and in >> memory from dotnet library, where its layout in memory is known, and >> something you want exactly as it is, but in python? If so, you should be >> able to use the methods from the afore linked thread. If not, you?re >> probably stuck iterating somewhere to reformat it, no matter what. Which >> is probably why you got garbage back. I?m guessing the object returned >> from the dotnet file-format-library isn?t laid out right, as suggested in >> the afore referenced caveat. >> >> >> > On Oct 28, 2014, at 9:55 AM, Nikhil wrote: >> > >> > Hello, >> > Yeah, I read data from a file say at each node and each time step, but >> when i try to use Marshal approach i get gibberish but when i use simple >> iter i get correct values. i have been trying the approach used in example >> in the previous post and that example makes sense but it doesnt make sense >> when i use it in my case. I am right now assigning it to a variable, i am >> now thinking of exploring the possibility of saving data to a dot net array >> maybe using System.Array and saving data to it but not sure if that even >> make sense. >> > >> > Sent from my iPhone >> >> _________________________________________________ >> Python.NET mailing list - PythonDotNet at python.org >> https://mail.python.org/mailman/listinfo/pythondotnet >> > > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet > -- Regards Nikhil ------------------------------------------------------------------- Big whirls have little whirls, Which feed on their velocity, And little whirls have lesser whirls, And so on to viscosity (Richardson, 1922) -------------- next part -------------- An HTML attachment was scrubbed... URL: