From savages at mozapps.org Fri Jun 4 03:15:58 2010 From: savages at mozapps.org (Shaun Savage) Date: Fri, 04 Jun 2010 09:15:58 +0800 Subject: [capi-sig] set default encoding in C program Message-ID: <4C0853CE.2030101@mozapps.org> I have to have Unicode strings in my program. Trying to set the default encoding to utf-8 is a pain. (another *RANT*). the way i was thinking about doing it is create a sitecustomization.py in the directory that the executable is in. That does not work. I can not change site.py or sitecustomization.py. How do you set the default encoding to 'utf-8'? shaun From ideasman42 at gmail.com Fri Jun 4 08:53:22 2010 From: ideasman42 at gmail.com (Campbell Barton) Date: Fri, 4 Jun 2010 08:53:22 +0200 Subject: [capi-sig] set default encoding in C program In-Reply-To: <4C0853CE.2030101@mozapps.org> References: <4C0853CE.2030101@mozapps.org> Message-ID: Could you set the environment variable in C before initializing python? http://docs.python.org/py3k/using/cmdline.html#envvar-PYTHONIOENCODING On Fri, Jun 4, 2010 at 3:15 AM, Shaun Savage wrote: > I have to have Unicode strings in my program. Trying to set the default > encoding to utf-8 is a pain. (another *RANT*). > > the way i was thinking about doing it is create a sitecustomization.py ?in > the directory that the executable is in. ?That does not work. I can not > change site.py or sitecustomization.py. > > How do you set the default encoding to 'utf-8'? > > shaun > > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > > -- - Campbell From rhamph at gmail.com Fri Jun 4 11:02:21 2010 From: rhamph at gmail.com (Adam Olsen) Date: Fri, 4 Jun 2010 03:02:21 -0600 Subject: [capi-sig] set default encoding in C program In-Reply-To: <4C0853CE.2030101@mozapps.org> References: <4C0853CE.2030101@mozapps.org> Message-ID: On Thu, Jun 3, 2010 at 19:15, Shaun Savage wrote: > I have to have Unicode strings in my program. Trying to set the default > encoding to utf-8 is a pain. (another *RANT*). > > the way i was thinking about doing it is create a sitecustomization.py ?in > the directory that the executable is in. ?That does not work. I can not > change site.py or sitecustomization.py. > > How do you set the default encoding to 'utf-8'? You don't change it. You wish it didn't exist and explicitly decode when data enters or leaves your program. What I suspect you're doing is creating unicode constants from C, so you're working from a UTF-8 encoded source file. I believe PyUnicode_DecodeUTF8() is your best bet there, even if you do have to explicitly pass the size. If you're not dealing with constants then it's not a big deal, and in fact you want an explicit size (as NUL does NOT indicate end of string.) From savages at mozapps.org Sat Jun 5 10:45:37 2010 From: savages at mozapps.org (Shaun Savage) Date: Sat, 05 Jun 2010 16:45:37 +0800 Subject: [capi-sig] PyThreads and pthreads Message-ID: <4C0A0EB1.5020109@mozapps.org> Next problems, (and hope last) I have started some Python threads, and want to wait for them to finish. I wrote this program to demonstrate my problem. I am on a 64 bit machine, Ubuntu. I tried playing around with GIL state but it does not change anything, so it is commented out. when using pthreads it works (of course) when I use python threads the sleep does not work and the threads all exit on the _join From savages at mozapps.org Sat Jun 5 12:05:03 2010 From: savages at mozapps.org (Shaun Savage) Date: Sat, 05 Jun 2010 18:05:03 +0800 Subject: [capi-sig] capi-sig Digest, Vol 35, Issue 4 In-Reply-To: References: Message-ID: <4C0A214F.1030408@mozapps.org> sorry about the attachment On 06/05/2010 06:00 PM, capi-sig-request at python.org wrote: > Next problems, (and hope last) > > I have started some Python threads, and want to wait for them to finish. > > I wrote this program to demonstrate my problem. I am on a 64 bit > machine, Ubuntu. > > I tried playing around with GIL state but it does not change anything, > so it is commented out. > > when using pthreads it works (of course) > when I use python threads the sleep does not work and the threads all > exit on the _join > > ------------------------------ > > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > > > End of capi-sig Digest, Vol 35, Issue 4 > *************************************** > #include #include #include #include typedef struct _thrdStruct { int num; pthread_t self; } thrdStruct; void* thrdFunc( void *p ) { thrdStruct *thrd = ( thrdStruct* ) p; thrd->self = pthread_self(); //PyGILState_STATE gstate = PyGILState_Ensure(); printf( "THRD %d\n", thrd->num); //Py_BEGIN_ALLOW_THREADS sleep(15); //Py_END_ALLOW_THREADS //PyGILState_Release(gstate); return NULL; } int main( int argc, char *argv[] ) { thrdStruct thrds[10]; pthread_t self[10] = {0}; int rv, i; void *stat; PyGILState_STATE gstate; Py_Initialize(); PyEval_InitThreads(); gstate = PyGILState_Ensure(); for (i=0;i<10;i++) { thrds[i].num = i; rv = PyThread_start_new_thread( thrdFunc, &thrds[i] ); //rv = pthread_create( &self[i], NULL, thrdFunc, &thrds[i] ); } PyGILState_Release(gstate); //Py_BEGIN_ALLOW_THREADS sleep(1); //Py_END_ALLOW_THREADS for (i=0;i<10;i++) { if ( self[i] != thrds[i].self ) printf( "%d %x %x\n", i, self[i], thrds[i].self); } for (i=0;i<10;i++) { rv = pthread_join( thrds[i].self, &stat ); } Py_Finalize(); } From swapnil.st at gmail.com Thu Jun 10 16:08:40 2010 From: swapnil.st at gmail.com (Swapnil Talekar) Date: Thu, 10 Jun 2010 19:38:40 +0530 Subject: [capi-sig] set default encoding in C program In-Reply-To: <4C0853CE.2030101@mozapps.org> References: <4C0853CE.2030101@mozapps.org> Message-ID: You could try sys.setdefaultencoding() On Fri, Jun 4, 2010 at 6:45 AM, Shaun Savage wrote: > I have to have Unicode strings in my program. Trying to set the default > encoding to utf-8 is a pain. (another *RANT*). > > the way i was thinking about doing it is create a sitecustomization.py in > the directory that the executable is in. That does not work. I can not > change site.py or sitecustomization.py. > > How do you set the default encoding to 'utf-8'? > > shaun > > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > > From davegb at pobox.com Thu Jun 10 20:59:44 2010 From: davegb at pobox.com (Dave Brotherstone) Date: Thu, 10 Jun 2010 20:59:44 +0200 Subject: [capi-sig] PyRun_InteractiveOne from string? Message-ID: Hi, I hope I'm asking this in the right place... I've embedded Python in an application, and I'm now trying to emulate a "console" for immediate commands. I can see PyRun_InteractiveOneFlags and the variants are used to send single statements, but there doesn't seem to be any variants that take a string, only a FILE*. I'm in a windows app, so there's no console. (I actually have effectively two text boxes, one multi-line read only for stdout/stderr, and one input.), I know the prompt would be in the wrong place, but that's no biggie. PyRun_String does almost what I want, except that it fails on multiline statements (if ... : etc). I just wanted to check if there's an API I've missed, or, whether anyone can recommend an approach. I think I could either: a) Emulate the console through an FILE* - not sure quite how this works in Windows, but I'm sure it's possible b) Copy what PyRun_InteractiveOneFlags() does, just replacing the PyParser_ASTFromFile with PyParser_ASTFromString. Only problem here is that I seem to need mod_ty, and not sure whether it's a good idea to use something that looks like an internal type. Many thanks for any comments. Dave. From ideasman42 at gmail.com Thu Jun 10 21:29:52 2010 From: ideasman42 at gmail.com (Campbell Barton) Date: Thu, 10 Jun 2010 21:29:52 +0200 Subject: [capi-sig] PyRun_InteractiveOne from string? In-Reply-To: References: Message-ID: In blender 2.5 we have an interactive console with multi-line support and auto complete. Demo of using the console http://sites.google.com/site/satishgoda/blender/learningblender25/introduction-to-blender-python-api Heres the script which interfaces python and blenders generic console api, look at how it uses the 'code' module. http://www.google.com/codesearch/p?hl=en#J_CGftyjihw/release/scripts/op/console_python.py&q=console_python&sa=N&cd=1&ct=rc We do our own line editing functionality but basically you can use the code module like this import code namespace = {'__builtins__': __builtins__} code.InteractiveConsole(namespace) is_multiline = console.push(line_exec) # <--- this is the line you need to run in a loop and give user input to. even though you mention not having a console, for the record that can be done like this... PyRun_String("__import__('code').interact()", Py_eval_input, namespace, namespace); On Thu, Jun 10, 2010 at 8:59 PM, Dave Brotherstone wrote: > Hi, > > I hope I'm asking this in the right place... ?I've embedded Python in an > application, and I'm now trying to emulate a "console" for immediate > commands. ?I can see PyRun_InteractiveOneFlags and the variants are used to > send single statements, but there doesn't seem to be any variants that take > a string, only a FILE*. ?I'm in a windows app, so there's no console. ?(I > actually have effectively two text boxes, one multi-line read only for > stdout/stderr, and one input.), I know the prompt would be in the wrong > place, but that's no biggie. > > PyRun_String does almost what I want, except that it fails on multiline > statements (if ... : etc). > > I just wanted to check if there's an API I've missed, or, whether anyone can > recommend an approach. > > I think I could either: > a) Emulate the console through an FILE* - not sure quite how this works in > Windows, but I'm sure it's possible > b) Copy what PyRun_InteractiveOneFlags() does, just replacing > the PyParser_ASTFromFile with PyParser_ASTFromString. ?Only problem here is > that I seem to need mod_ty, and not sure whether it's a good idea to use > something that looks like an internal type. > > Many thanks for any comments. > > Dave. > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > -- - Campbell From davegb at pobox.com Fri Jun 11 22:16:18 2010 From: davegb at pobox.com (Dave Brotherstone) Date: Fri, 11 Jun 2010 22:16:18 +0200 Subject: [capi-sig] PyRun_InteractiveOne from string? In-Reply-To: References: Message-ID: Thanks so much.. that looks like exactly what I want to do. Looks like you've done a fantastic job on the integration there - makes my weedy little console look very plain :) Just need to work out why boost::python won't exec code.InteractiveConsole() now :) Thanks again. Dave. On Thu, Jun 10, 2010 at 9:29 PM, Campbell Barton wrote: > In blender 2.5 we have an interactive console with multi-line support > and auto complete. > Demo of using the console > > http://sites.google.com/site/satishgoda/blender/learningblender25/introduction-to-blender-python-api > > Heres the script which interfaces python and blenders generic console > api, look at how it uses the 'code' module. > > http://www.google.com/codesearch/p?hl=en#J_CGftyjihw/release/scripts/op/console_python.py&q=console_python&sa=N&cd=1&ct=rc > > We do our own line editing functionality but basically you can use the > code module like this > import code > namespace = {'__builtins__': __builtins__} > code.InteractiveConsole(namespace) > is_multiline = console.push(line_exec) # <--- this is the line you > need to run in a loop and give user input to. > > even though you mention not having a console, for the record that can > be done like this... > PyRun_String("__import__('code').interact()", Py_eval_input, > namespace, namespace); > > On Thu, Jun 10, 2010 at 8:59 PM, Dave Brotherstone > wrote: > > Hi, > > > > I hope I'm asking this in the right place... I've embedded Python in an > > application, and I'm now trying to emulate a "console" for immediate > > commands. I can see PyRun_InteractiveOneFlags and the variants are used > to > > send single statements, but there doesn't seem to be any variants that > take > > a string, only a FILE*. I'm in a windows app, so there's no console. (I > > actually have effectively two text boxes, one multi-line read only for > > stdout/stderr, and one input.), I know the prompt would be in the wrong > > place, but that's no biggie. > > > > PyRun_String does almost what I want, except that it fails on multiline > > statements (if ... : etc). > > > > I just wanted to check if there's an API I've missed, or, whether anyone > can > > recommend an approach. > > > > I think I could either: > > a) Emulate the console through an FILE* - not sure quite how this works > in > > Windows, but I'm sure it's possible > > b) Copy what PyRun_InteractiveOneFlags() does, just replacing > > the PyParser_ASTFromFile with PyParser_ASTFromString. Only problem here > is > > that I seem to need mod_ty, and not sure whether it's a good idea to use > > something that looks like an internal type. > > > > Many thanks for any comments. > > > > Dave. > > _______________________________________________ > > capi-sig mailing list > > capi-sig at python.org > > http://mail.python.org/mailman/listinfo/capi-sig > > > > > > -- > - Campbell > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > From python-dev at code2develop.com Tue Jun 15 10:26:33 2010 From: python-dev at code2develop.com (F van der Meeren) Date: Tue, 15 Jun 2010 10:26:33 +0200 Subject: [capi-sig] Linking with a static python library. Message-ID: <7C201BDC-2A66-4780-B3CC-72A91E5A8780@code2develop.com> Hi, I am trying to figure out, what files to copy with my app so I am able to initialize the python runtime. Where can I find information about this? Thank you, Filip From mal at egenix.com Tue Jun 15 11:10:54 2010 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 15 Jun 2010 11:10:54 +0200 Subject: [capi-sig] Linking with a static python library. In-Reply-To: <7C201BDC-2A66-4780-B3CC-72A91E5A8780@code2develop.com> References: <7C201BDC-2A66-4780-B3CC-72A91E5A8780@code2develop.com> Message-ID: <4C17439E.4030901@egenix.com> F van der Meeren wrote: > Hi, > > I am trying to figure out, what files to copy with my app so I am able to initialize the python runtime. > Where can I find information about this? This should get you started: http://docs.python.org/extending/embedding.html Apart from the C runtime, you'll also need to ship the Python stdlib, e.g. in a ZIP file. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 15 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 33 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ideasman42 at gmail.com Tue Jun 15 11:53:16 2010 From: ideasman42 at gmail.com (Campbell Barton) Date: Tue, 15 Jun 2010 11:53:16 +0200 Subject: [capi-sig] PyRun_InteractiveOne from string? In-Reply-To: References: Message-ID: I should have mentioned the console I linked to is not thread safe, I asked in #python about this and apparently there are 2 ways to work around this. - use a context manager to work out which thread is taking to the overridden stdout. - override print() not to use sys.stdout, since the consoles "write" function can already redirect exception output. Id like to make this thread safe but for now its not a priority for me. >From the script I linked this looks like its done entirely in python, this is because we have a py/c api so python has access to the command line and can add output to the display, rather then having C call python directly but both ways can work. On Fri, Jun 11, 2010 at 10:16 PM, Dave Brotherstone wrote: > Thanks so much.. that looks like exactly what I want to do. ?Looks like > you've done a fantastic job on the integration there - makes my weedy little > console look very plain :) > > Just need to work out why boost::python won't exec code.InteractiveConsole() > now :) > > Thanks again. > Dave. > > > > On Thu, Jun 10, 2010 at 9:29 PM, Campbell Barton wrote: > >> In blender 2.5 we have an interactive console with multi-line support >> and auto complete. >> Demo of using the console >> >> http://sites.google.com/site/satishgoda/blender/learningblender25/introduction-to-blender-python-api >> >> Heres the script which interfaces python and blenders generic console >> api, look at how it uses the 'code' module. >> >> http://www.google.com/codesearch/p?hl=en#J_CGftyjihw/release/scripts/op/console_python.py&q=console_python&sa=N&cd=1&ct=rc >> >> We do our own line editing functionality but basically you can use the >> code module like this >> ?import code >> ?namespace = {'__builtins__': __builtins__} >> ?code.InteractiveConsole(namespace) >> ?is_multiline = console.push(line_exec) # <--- this is the line you >> need to run in a loop and give user input to. >> >> even though you mention not having a console, for the record that can >> be done like this... >> PyRun_String("__import__('code').interact()", Py_eval_input, >> namespace, namespace); >> >> On Thu, Jun 10, 2010 at 8:59 PM, Dave Brotherstone >> wrote: >> > Hi, >> > >> > I hope I'm asking this in the right place... ?I've embedded Python in an >> > application, and I'm now trying to emulate a "console" for immediate >> > commands. ?I can see PyRun_InteractiveOneFlags and the variants are used >> to >> > send single statements, but there doesn't seem to be any variants that >> take >> > a string, only a FILE*. ?I'm in a windows app, so there's no console. ?(I >> > actually have effectively two text boxes, one multi-line read only for >> > stdout/stderr, and one input.), I know the prompt would be in the wrong >> > place, but that's no biggie. >> > >> > PyRun_String does almost what I want, except that it fails on multiline >> > statements (if ... : etc). >> > >> > I just wanted to check if there's an API I've missed, or, whether anyone >> can >> > recommend an approach. >> > >> > I think I could either: >> > a) Emulate the console through an FILE* - not sure quite how this works >> in >> > Windows, but I'm sure it's possible >> > b) Copy what PyRun_InteractiveOneFlags() does, just replacing >> > the PyParser_ASTFromFile with PyParser_ASTFromString. ?Only problem here >> is >> > that I seem to need mod_ty, and not sure whether it's a good idea to use >> > something that looks like an internal type. >> > >> > Many thanks for any comments. >> > >> > Dave. >> > _______________________________________________ >> > capi-sig mailing list >> > capi-sig at python.org >> > http://mail.python.org/mailman/listinfo/capi-sig >> > >> >> >> >> -- >> - Campbell >> _______________________________________________ >> capi-sig mailing list >> capi-sig at python.org >> http://mail.python.org/mailman/listinfo/capi-sig >> > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > -- - Campbell From davegb at pobox.com Tue Jun 15 18:23:04 2010 From: davegb at pobox.com (Dave Brotherstone) Date: Tue, 15 Jun 2010 18:23:04 +0200 Subject: [capi-sig] PyRun_InteractiveOne from string? In-Reply-To: References: Message-ID: Thanks for this. It's working perfectly now. Funnily enough threading was the next thing on the list, so the (non-console) scripts run in a separate thread, allowing them to be stopped by the UI. You don't happen to know if PyErr_SetString is thread safe, to "break" the currently running script? I'll have to think about whether taking care of separate threads redirecting sys.stdout is important - I'm not sure that it is in our environment. Cheers, Dave. On Tue, Jun 15, 2010 at 11:53 AM, Campbell Barton wrote: > I should have mentioned the console I linked to is not thread safe, I > asked in #python about this and apparently there are 2 ways to work > around this. > - use a context manager to work out which thread is taking to the > overridden stdout. > - override print() not to use sys.stdout, since the consoles "write" > function can already redirect exception output. > > Id like to make this thread safe but for now its not a priority for me. > > From the script I linked this looks like its done entirely in python, > this is because we have a py/c api so python has access to the command > line and can add output to the display, rather then having C call > python directly but both ways can work. > > On Fri, Jun 11, 2010 at 10:16 PM, Dave Brotherstone > wrote: > > Thanks so much.. that looks like exactly what I want to do. Looks like > > you've done a fantastic job on the integration there - makes my weedy > little > > console look very plain :) > > > > Just need to work out why boost::python won't exec > code.InteractiveConsole() > > now :) > > > > Thanks again. > > Dave. > > > > > > > > On Thu, Jun 10, 2010 at 9:29 PM, Campbell Barton >wrote: > > > >> In blender 2.5 we have an interactive console with multi-line support > >> and auto complete. > >> Demo of using the console > >> > >> > http://sites.google.com/site/satishgoda/blender/learningblender25/introduction-to-blender-python-api > >> > >> Heres the script which interfaces python and blenders generic console > >> api, look at how it uses the 'code' module. > >> > >> > http://www.google.com/codesearch/p?hl=en#J_CGftyjihw/release/scripts/op/console_python.py&q=console_python&sa=N&cd=1&ct=rc > >> > >> We do our own line editing functionality but basically you can use the > >> code module like this > >> import code > >> namespace = {'__builtins__': __builtins__} > >> code.InteractiveConsole(namespace) > >> is_multiline = console.push(line_exec) # <--- this is the line you > >> need to run in a loop and give user input to. > >> > >> even though you mention not having a console, for the record that can > >> be done like this... > >> PyRun_String("__import__('code').interact()", Py_eval_input, > >> namespace, namespace); > >> > >> On Thu, Jun 10, 2010 at 8:59 PM, Dave Brotherstone > >> wrote: > >> > Hi, > >> > > >> > I hope I'm asking this in the right place... I've embedded Python in > an > >> > application, and I'm now trying to emulate a "console" for immediate > >> > commands. I can see PyRun_InteractiveOneFlags and the variants are > used > >> to > >> > send single statements, but there doesn't seem to be any variants that > >> take > >> > a string, only a FILE*. I'm in a windows app, so there's no console. > (I > >> > actually have effectively two text boxes, one multi-line read only for > >> > stdout/stderr, and one input.), I know the prompt would be in the > wrong > >> > place, but that's no biggie. > >> > > >> > PyRun_String does almost what I want, except that it fails on > multiline > >> > statements (if ... : etc). > >> > > >> > I just wanted to check if there's an API I've missed, or, whether > anyone > >> can > >> > recommend an approach. > >> > > >> > I think I could either: > >> > a) Emulate the console through an FILE* - not sure quite how this > works > >> in > >> > Windows, but I'm sure it's possible > >> > b) Copy what PyRun_InteractiveOneFlags() does, just replacing > >> > the PyParser_ASTFromFile with PyParser_ASTFromString. Only problem > here > >> is > >> > that I seem to need mod_ty, and not sure whether it's a good idea to > use > >> > something that looks like an internal type. > >> > > >> > Many thanks for any comments. > >> > > >> > Dave. > >> > _______________________________________________ > >> > capi-sig mailing list > >> > capi-sig at python.org > >> > http://mail.python.org/mailman/listinfo/capi-sig > >> > > >> > >> > >> > >> -- > >> - Campbell > >> _______________________________________________ > >> capi-sig mailing list > >> capi-sig at python.org > >> http://mail.python.org/mailman/listinfo/capi-sig > >> > > _______________________________________________ > > capi-sig mailing list > > capi-sig at python.org > > http://mail.python.org/mailman/listinfo/capi-sig > > > > > > -- > - Campbell >