From xionglonghui at gmail.com Wed Mar 4 07:11:26 2015 From: xionglonghui at gmail.com (Longhui Xiong) Date: Wed, 4 Mar 2015 14:11:26 +0800 Subject: [Python.NET] About use DotNet threading lib Message-ID: Hi there, I am using python for DotNet, I have to say, it is great. As we know, cpython has the Global Interpreter Lock (GIL) on multi-threading. So it can not use multi-core. So I have one question, if I import CLR and use System.Threading on python for .net, but don't use cpython threading, My question: Does it still has GIL on multi-threading if I use DotNet threading lib? Can it use multi-core use DotNet threading lib. I know IronPython has no GIL and multi-threaded code can use multi core processors. So I pay attention this issue, on python for dotnet, if I use DotNet threading lib instead of cPython threading lib, Can multi-threaded code use multi core, it still has GIL? I look forward to your reply. Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff at coderforlife.com Wed Mar 4 19:34:26 2015 From: jeff at coderforlife.com (Jeffrey Bush) Date: Wed, 4 Mar 2015 10:34:26 -0800 Subject: [Python.NET] About use DotNet threading lib In-Reply-To: References: Message-ID: You seem to be slightly misinformed about the cPython GIL. While it is true that it prevents two Python statements from being interpreted at the same time, many "long-running" functions automatically release the GIL while they are being executed. For example, when reading and writing a file the GIL is released, allowing other threads to do things at that time. Many other built in functions also release the GIL: time.sleep(), select.select(), lock.acquire() and in general anything else that can block. Also, the ctypes library releases the GIL in when calling native functions (except when using ctypes.PyDLL). Many third party libraries also do this as well (for example, numpy/scipy does this all the time). I have written plenty of multi-threaded Python programs that make full use of all the cores on a machine - maxing them all out. If your code is really heavy on the Python statements that cannot be executed at the same time one solution may be to spawn multiple Python interpreters so that they can execute statements at the same time. The downside is that it is more difficult to share data between "threads". See https://docs.python.org/2/library/multiprocessing.html which handles a lot of this for you and is designed to be a replacement for threads. Now on to your question. If you start a .NET thread it will not be under the influence of the Python GIL since it is running .NET code and not Python. The .NET code knows nothing about the GIL and Python has no ability to control the execution of individual .NET statements (which don't exist once compiled anyways). However, if your .NET code runs a Python statement, those statements will be subject to the GIL. So the basic idea is that no two Python statements may be interpreted at the same time using the same interpreter, regardless of where they came from, under cPython. However multiple Python statements may be executed at the same time (if at least one is designed to release the GIL). And any non-Python code may be run at the same time without influencing or being influencing the GIL. Just for an explanation of why the GIL exists in the first place: many of the built-in classes (e.g. dict) are not thread-safe on their own and if used without the GIL in a multi-thread way would cause lots of problems. Other languages address this in other ways, e.g. .NET has Dictionary vs ConcurrentDictionary. In Python they addressed it by making the default situation that all statements are thread-safe instead of using explicitly a different class. There are some other reasons why it exists as well, but this is getting long. Jeff On Tue, Mar 3, 2015 at 10:11 PM, Longhui Xiong wrote: > Hi there, > > I am using python for DotNet, I have to say, it is great. > > As we know, cpython has the Global Interpreter Lock (GIL) on > multi-threading. So it can not use multi-core. > > So I have one question, if I import CLR and use System.Threading on python > for .net, but don't use cpython threading, > > My question: Does it still has GIL on multi-threading if I use DotNet > threading lib? Can it use multi-core use DotNet threading lib. > > I know IronPython has no GIL and multi-threaded code can use multi core > processors. So I pay attention this issue, on python for dotnet, if I use > DotNet threading lib instead of cPython threading lib, Can multi-threaded > code use multi core, it still has GIL? > > I look forward to your reply. > > Regards > > _________________________________________________ > Python.NET mailing list - PythonDotNet at python.org > https://mail.python.org/mailman/listinfo/pythondotnet > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cameron.hayne at introspect.ca Thu Mar 26 01:49:28 2015 From: cameron.hayne at introspect.ca (Cameron Hayne) Date: Wed, 25 Mar 2015 20:49:28 -0400 Subject: [Python.NET] changes to Python.Runtime.Converter and Python.Runtime.Object Message-ID: I have been using the version of PythonDotNet from https://github.com/renshawbay/pythonnet for embedding - i.e. to call Python code in C# programs. I have made a few changes to the source code in order to make using Python objects in C# easier, or to fix problems. Here are the changes: 1) In the source file ?converter.cs? which defines the class Python.Runtime.Converter, I added the following to the switch case ?TypeCode.Object? in the ?ToPython? function: case TypeCode.Object: // The following ?if? clause is needed (as suggested by Patrick Stewart) to make the examples in the README work. if (value is IEnumerable) { using (var resultlist = new PyList()) { foreach (object o in (IEnumerable)value) { Type oType = o.GetType(); TypeCode oTc = Type.GetTypeCode(oType); // The following 'if' clause for Object entries is needed so that the entries // in lists of Python objects don't lose their types. // I.e. so that if you do something like: // dynamic obj1 = myPythonClass(); // List myList = new List { obj1 }; // myPythonFunc(myList); // then the type of myList[0] inside myPythonFunc will be the same as the type of obj1 // whereas the 'else' clause would make it be of type Python.Runtime.PyObject if (oTc == TypeCode.Object) { resultlist.Append((PyObject)o); } else { using (var p = new PyObject(ToPython(o, oType))) resultlist.Append(p); } } Runtime.Incref(resultlist.Handle); return resultlist.Handle; } } 2) In the source file ?pyobject.cs? which defines the class Python.Runtime.PyObject, I replaced the line: this.SetAttr(binder.Name, (PyObject)value); in the ?if (this.HasAttr(binder.Name))? clause in the function ?TrySetMember? with the following: // Use ?ToPython? if an exception occurs when casting to (PyObject) // Without this, get exception e.g. when assign a C# String to a Python member // like: myPyObj.name = name; PyObject pyValue; try { pyValue = (PyObject)value; } catch (Exception) { pyValue = value.ToPython(); } this.SetAttr(binder.Name, pyValue); I would be interested in any comments about these changes - especially if there are better ways of fixing the problems. I am also interested to hear if anyone has other improvements to suggest. -- Cameron Hayne cameron.hayne at introspect.ca