From AndiDog at web.de Wed Oct 14 22:05:04 2009 From: AndiDog at web.de (Andreas Sommer) Date: Wed, 14 Oct 2009 22:05:04 +0200 Subject: [capi-sig] C integration with multithreaded Python script Message-ID: <4AD62EF0.2040602@web.de> Hope this is the right mailing list, please complain if not ;) I'm currently experimenting with the integration of Python (2.6) scripts into my application, and am using the following script for testing: *import* threading, time *def* handler(): print("a") time.sleep(0.95) print("b") t = threading.Thread(target = handler) t.start() And in C++, it's this code: Py_SetProgramName("test"); Py_Initialize(); *if*(!Py_IsInitialized()) *throw* std::exception(); PyObject *module = PyImport_AddModule("__main__"); PyObject *dict = PyModule_GetDict(module); printf("1\n"); strcpy(script, "import threading, time\n" "def handler():\n" " print(\"a\")\n" " time.sleep(0.95)\n" " print(\"b\")\n" "t = threading.Thread(target = handler)\n" "t.start()\n"); PyObject *obj = PyRun_String(script, Py_file_input, dict, dict); printf("2\n"); PyErr_Print(); Py_Finalize(); When executing, the output is 1 a2 as opposed to the expected 1 a b 2 It seems to me that when the main execution path (after t.start()) finishes, the other thread (called t) is canceled?! Adding t.join() to the end of the script gives the expected output, but isn't it possible to let PyRun_xxx wait for all remaining threads to end? I know the Python interpreter does this, but I didn't get much information from its source code (it seems to run scripts with the runpy module). Best regards From george_edison55 at hotmail.com Tue Oct 20 09:48:12 2009 From: george_edison55 at hotmail.com (Nathan Osman) Date: Tue, 20 Oct 2009 00:48:12 -0700 Subject: [capi-sig] Couple of Questions About Statically Linking the Interpreter Message-ID: Im kind of new to the Python/C api and I had a couple q's about the interpreter. 1. If i statically link the libpython31.a library to my app are there any other files i need to include for the end users? Im using Windows, btw. 2. How do you get the interpreter output from the console when you are running within a gui environment? Can it be redirected? Thanks, George _________________________________________________________________ New! Faster Messenger access on the new MSN homepage http://go.microsoft.com/?linkid=9677406 From python_capi at behnel.de Tue Oct 20 10:09:28 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 20 Oct 2009 10:09:28 +0200 Subject: [capi-sig] Couple of Questions About Statically Linking the Interpreter In-Reply-To: References: Message-ID: <4ADD7038.7050305@behnel.de> Nathan Osman wrote: > Im kind of new to the Python/C api and I had a couple q's about the interpreter. > > 1. If i statically link the libpython31.a library to my app are there > any other files i need to include for the end users? You will most likely need the standard library as well. The freeze script in Python/Tools/ (source distro) might help here. There's also py2exe, but that creates a stand-alone application, not a linkable binary (AFAIK). > 2. How do you get the interpreter output from the console when you are > running within a gui environment? Can it be redirected? sys.stdout = ... Stefan From ideasman42 at gmail.com Tue Oct 20 15:02:54 2009 From: ideasman42 at gmail.com (Campbell Barton) Date: Tue, 20 Oct 2009 15:02:54 +0200 Subject: [capi-sig] Couple of Questions About Statically Linking the Interpreter In-Reply-To: References: Message-ID: <7c1ab96d0910200602s77f5c8a0r78be776de4f5910c@mail.gmail.com> for python 2.x you can get away only with statically linked library, but then you miss modules like math, struct and os If you want these modules you can compile some of these into the library you can build python with a modified config -- quick howto -- edit ./Modules/Setup uncomment "#*shared*" around line 152 and rename to "*static*" The following modules are built into libpython2.5.a uncomment: array, math, time, operator, itertools, cStringIO, cPickle, zlib, _struct, _weakref, _random, binascii, collections, fcntl, spwd, grp, select -- Or you can bundle python25.zip with your application and include all modules in this, however youll need zlib either compiled statically or include the module unzipped in the dir. Blender2.49 for example in win32 has python26.zip and zlib.pyd in the current directory so python can extract the zip and open the modules... As for python 3.x, It cant even start unless it has some external modules Unless there is some trick I dont know of, youll get this with a static linked py3.1 and no modules... --- Fatal Python error: Py_Initialize: can't initialize sys standard streams ImportError: No module named encodings.utf_8 --- Heres some info I wrote about how to bundle python3.1 with blender. http://wiki.blender.org/index.php/BlenderDev/Blender2.5/PythonAPI_31 On Tue, Oct 20, 2009 at 9:48 AM, Nathan Osman wrote: > Im kind of new to the Python/C api and I had a couple q's about the interpreter. > > 1. If i statically link the libpython31.a library to my app are there any other files i need to include for the end users? Im using Windows, btw. > 2. How do you get the interpreter output from the console when you are running within a gui environment? Can it be redirected? > > Thanks, > George > > _________________________________________________________________ > New! Faster Messenger access on the new MSN homepage > http://go.microsoft.com/?linkid=9677406 > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > -- - Campbell From ideasman42 at gmail.com Tue Oct 20 22:11:25 2009 From: ideasman42 at gmail.com (Campbell Barton) Date: Tue, 20 Oct 2009 22:11:25 +0200 Subject: [capi-sig] Couple of Questions About Statically Linking the Interpreter In-Reply-To: References: <7c1ab96d0910200602s77f5c8a0r78be776de4f5910c@mail.gmail.com> Message-ID: <7c1ab96d0910201311r40d63d63n50fb65648df198b4@mail.gmail.com> There are a few ways to set this path from C before starting python setenv("PYTHONHOME", somepath, 1); or... Py_SetPythonHome(py_path_bundle_wchar); Heres a function that blender uses to setup python3.1's path, there are a few blender functions here but should still be useful. the #if0's part works too but I preferred to set the home folder directly. /* must be called before Py_Initialize */ void BPY_start_python_path(void) { char *py_path_bundle= BLI_gethome_folder("python", BLI_GETHOME_ALL); if(py_path_bundle==NULL) return; /* set the environment path */ printf("found bundled python: %s\n", py_path_bundle); #if 0 BLI_setenv("PYTHONHOME", py_path_bundle); BLI_setenv("PYTHONPATH", py_path_bundle); #endif { static wchar_t py_path_bundle_wchar[FILE_MAXDIR]; mbstowcs(py_path_bundle_wchar, py_path_bundle, FILE_MAXDIR); Py_SetPythonHome(py_path_bundle_wchar); } } On Tue, Oct 20, 2009 at 8:34 PM, Nathan Osman wrote: > Thanks for your help. So if I bundle everything in the Lib directory with > the application what function would I call to inform Python where the > modules are located before I call Py_Initialize? > > Thanks, > Nathan > >> Date: Tue, 20 Oct 2009 15:02:54 +0200 >> From: ideasman42 at gmail.com >> To: capi-sig at python.org >> Subject: Re: [capi-sig] Couple of Questions About Statically Linking the >> Interpreter >> >> for python 2.x you can get away only with statically linked library, >> but then you miss modules like math, struct and os >> If you want these modules you can compile some of these into the >> library you can build python with a modified config >> >> -- quick howto -- >> edit ./Modules/Setup >> uncomment "#*shared*" around line 152 and rename to "*static*" The >> following modules are built into libpython2.5.a >> uncomment: array, math, time, operator, itertools, cStringIO, cPickle, >> zlib, _struct, _weakref, _random, binascii, collections, fcntl, spwd, >> grp, select >> -- >> >> Or you can bundle python25.zip with your application and include all >> modules in this, however youll need zlib either compiled statically or >> include the module unzipped in the dir. >> >> Blender2.49 for example in win32 has python26.zip and zlib.pyd in the >> current directory so python can extract the zip and open the >> modules... >> >> As for python 3.x, It cant even start unless it has some external modules >> >> Unless there is some trick I dont know of, youll get this with a >> static linked py3.1 and no modules... >> --- >> Fatal Python error: Py_Initialize: can't initialize sys standard streams >> ImportError: No module named encodings.utf_8 >> --- >> >> Heres some info I wrote about how to bundle python3.1 with blender. >> >> http://wiki.blender.org/index.php/BlenderDev/Blender2.5/PythonAPI_31 >> >> On Tue, Oct 20, 2009 at 9:48 AM, Nathan Osman >> wrote: >> > Im kind of new to the Python/C api and I had a couple q's about the >> > interpreter. >> > >> > 1. If i statically link the libpython31.a library to my app are there >> > any other files i need to include for the end users? Im using Windows, btw. >> > 2. How do you get the interpreter output from the console when you are >> > running within a gui environment? Can it be redirected? >> > >> > Thanks, >> > George >> > >> > _________________________________________________________________ >> > New! Faster Messenger access on the new MSN homepage >> > http://go.microsoft.com/?linkid=9677406 >> > _______________________________________________ >> > capi-sig mailing list >> > capi-sig at python.org >> > http://mail.python.org/mailman/listinfo/capi-sig >> > >> >> >> >> -- >> - Campbell >> _______________________________________________ >> capi-sig mailing list >> capi-sig at python.org >> http://mail.python.org/mailman/listinfo/capi-sig > > ________________________________ > New! Hotmail sign-in on the MSN homepage. -- - Campbell From george_edison55 at hotmail.com Sun Oct 25 02:31:45 2009 From: george_edison55 at hotmail.com (Nathan Osman) Date: Sat, 24 Oct 2009 18:31:45 -0700 Subject: [capi-sig] Garbage Collection Error Message-ID: Everyone was so helpful with my last question, I think I'll ask another. I have an app that uses the Python/C api. It loads a few scripts, calls a few functions, and everything works fine. That is, until the app exits and calls Py_Finalize. Then the following error appears: (GetName is a function in one of the scripts) Exception AttributeError: "'module' object has no attribute 'GetName'" in 'garbage collection' ignored Fatal Python error: unexpected exception during garbage collection Then the app crashes. Im using Python 3.1 on Windows if that helps. Any advice would be appreciated. - George _________________________________________________________________ CDN College or University student? Get Windows 7 for only $39.99 before Jan 3! Buy it now! http://go.microsoft.com/?linkid=9691636 From AndiDog at web.de Sat Oct 31 21:10:36 2009 From: AndiDog at web.de (Andreas Sommer) Date: Sat, 31 Oct 2009 21:10:36 +0100 Subject: [capi-sig] C integration with multithreaded Python script In-Reply-To: <4AD62EF0.2040602@web.de> References: <4AD62EF0.2040602@web.de> Message-ID: <4AEC99BC.4080304@web.de> I still don't have an answer to this question. Meanwhile I've taken a look at the Python interpreter source code and couldn't find out how it manages to keep running while other threads exist. Taking the /RunModule/ function from the code doesn't help me - same problem: only the first few instructions of the background thread are executed and then Python execution ends. Andreas Sommer wrote: > Hope this is the right mailing list, please complain if not ;) > > I'm currently experimenting with the integration of Python (2.6) > scripts into my application, and am using the following script for > testing: > > *import* threading, time > > *def* handler(): > print("a") > time.sleep(0.95) > print("b") > > t = threading.Thread(target = handler) > t.start() > > > And in C++, it's this code: > > Py_SetProgramName("test"); > Py_Initialize(); > *if*(!Py_IsInitialized()) *throw* std::exception(); > > > PyObject *module = PyImport_AddModule("__main__"); > PyObject *dict = PyModule_GetDict(module); > > printf("1\n"); > > strcpy(script, "import threading, time\n" > "def handler():\n" > " print(\"a\")\n" > " time.sleep(0.95)\n" > " print(\"b\")\n" > "t = threading.Thread(target = handler)\n" > "t.start()\n"); > > PyObject *obj = PyRun_String(script, Py_file_input, dict, dict); > printf("2\n"); > > > PyErr_Print(); > Py_Finalize(); > > > When executing, the output is > > 1 > a2 > > as opposed to the expected > > 1 > a > b > 2 > > It seems to me that when the main execution path (after t.start()) > finishes, the other thread (called t) is canceled?! Adding t.join() to > the end of the script gives the expected output, but isn't it possible > to let PyRun_xxx wait for all remaining threads to end? I know the > Python interpreter does this, but I didn't get much information from > its source code (it seems to run scripts with the runpy module). > > > Best regards