From tismer at stackless.com Mon Feb 2 11:28:08 2009 From: tismer at stackless.com (Christian Tismer) Date: Mon, 02 Feb 2009 02:28:08 -0800 Subject: [pypy-dev] Psyco version 2 preview is available Message-ID: <4986CAB8.30604@stackless.com> This is not an official announcement of Psyco V2, yet. But everybody who is interested should check out http://codespeak.net:/svn/psyco/v2 and do some testing. The major improvement besides lots of builtins is the new generator support, which is disabled by default, due to an internal error in Psyco. To use and test generators, create preferences.py, following the instructions in setup.py. This software is considered asan alpha release. The final version should be expected before end of February. regards -- chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From kfh at mqsoftware.com Wed Feb 11 19:28:58 2009 From: kfh at mqsoftware.com (Kelly F. Hickel) Date: Wed, 11 Feb 2009 12:28:58 -0600 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and dbm Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> Hi all, I didn't see a pypy-user list, so I'm posting here. I came across PyPy while trying to find a way to speed up cvs2svn while converting a large CVS repo. I've pulled the latest via svn, got it built and have translated the interpreter, but cvs2svn using pypy-c fails with the output: debug: WARNING: library path not found, using compiled-in sys.path ERROR: cvs2svn uses the anydbm package, which depends on lower level dbm libraries. Your system has dumbdbm, with which cvs2svn is known to have problems. To use cvs2svn, you must install a Python dbm library other than dumbdbm or dbm. See http://python.org/doc/current/lib/module-anydbm.html for more information. Now, when we've run into this issue with python, someone just rebuilt python enabling built in libgdbm support, but I haven't found any way to do that or include some other dbm within PyPy. Since I'm not that familiar with python to start with, I'm hoping that I'm just missing something quite basic, and that some kind soul here can point me in the right direction... Should I be trying to translate cvs2svn instead of trying using pypy-c ? (this seems to involve building a target file or something). Am I just out of luck? -- Kelly F. Hickel Senior Product Architect MQSoftware, Inc. 952-345-8677 Office 952-345-8721 Fax kfh at mqsoftware.com www.mqsoftware.com Certified IBM SOA Specialty Your Full Service Provider for IBM WebSphere Learn more at www.mqsoftware.com From santagada at gmail.com Wed Feb 11 19:36:35 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 11 Feb 2009 16:36:35 -0200 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and dbm In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> Message-ID: <26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com> On Feb 11, 2009, at 4:28 PM, Kelly F. Hickel wrote: > Hi all, I didn't see a pypy-user list, so I'm posting here. > > I came across PyPy while trying to find a way to speed up cvs2svn > while > converting a large CVS repo. I've pulled the latest via svn, got it > built and have translated the interpreter, but cvs2svn using pypy-c > fails with the output: > debug: WARNING: library path not found, using compiled-in sys.path > ERROR: cvs2svn uses the anydbm package, which depends on lower level > dbm > libraries. Your system has dumbdbm, with which cvs2svn is known to > have > problems. To use cvs2svn, you must install a Python dbm library other > than > dumbdbm or dbm. See > http://python.org/doc/current/lib/module-anydbm.html > for more information. > > Now, when we've run into this issue with python, someone just rebuilt > python enabling built in libgdbm support, but I haven't found any > way to > do that or include some other dbm within PyPy. > > Since I'm not that familiar with python to start with, I'm hoping that > I'm just missing something quite basic, and that some kind soul here > can > point me in the right direction... > > Should I be trying to translate cvs2svn instead of trying using pypy- > c ? > (this seems to involve building a target file or something). > > Am I just out of luck? Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know would not give you any improvements in performance from CPython for this (maybe even psyco will not help). ps: Isn't cvs2svn a one time thing? -- Leonardo Santagada santagada at gmail.com From kfh at mqsoftware.com Wed Feb 11 19:41:47 2009 From: kfh at mqsoftware.com (Kelly F. Hickel) Date: Wed, 11 Feb 2009 12:41:47 -0600 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and dbm In-Reply-To: <26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com> References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> <26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com> Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com> > > I'm just missing something quite basic, and that some kind soul here > > can > > point me in the right direction... > > > > Should I be trying to translate cvs2svn instead of trying using pypy- > > c ? > > (this seems to involve building a target file or something). > > > > Am I just out of luck? > > Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know would > not give you any improvements in performance from CPython for this > (maybe even psyco will not help). > > ps: Isn't cvs2svn a one time thing? > > -- > Leonardo Santagada > santagada at gmail.com Thanks for the quick reply, I'll give it a try. I had looked at pysco back in June, but got the impression that I'd be better off with PyPy.... Yes, cvs2svn is a one time thing in principle, but I've been running it a number of times, trying different things, as a means of validating the switch to git (cvs2svn also does cvs2git). Since it takes 6.5 days on a 2.4ghz AMD with 32gb of ram to translate our repo, even a small tweak to the process is quite expensive.... -Kelly From kfh at mqsoftware.com Wed Feb 11 19:49:59 2009 From: kfh at mqsoftware.com (Kelly F. Hickel) Date: Wed, 11 Feb 2009 12:49:59 -0600 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com> References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com><26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com> <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com> Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com> > > > > Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know > would > > not give you any improvements in performance from CPython for this > > (maybe even psyco will not help). > > > > ps: Isn't cvs2svn a one time thing? > > > > -- > > Leonardo Santagada > > santagada at gmail.com > > Thanks for the quick reply, I'll give it a try. I had looked at pysco > back in June, but got the impression that I'd be better off with > PyPy.... > > Yes, cvs2svn is a one time thing in principle, but I've been running it > a number of times, trying different things, as a means of validating > the > switch to git (cvs2svn also does cvs2git). Since it takes 6.5 days on > a > 2.4ghz AMD with 32gb of ram to translate our repo, even a small tweak > to > the process is quite expensive.... > > -Kelly Ahh, yes, now I remember. Psyco ONLY works with 32 bit python, whereas I MUST use 64 bit because the memory footprint for cvs2svn with our repo is just too large for 32 bit processes. So, dead in the water again? -Kelly From alexandre.fayolle at logilab.fr Thu Feb 12 09:14:20 2009 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 12 Feb 2009 09:14:20 +0100 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com> References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com> <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com> Message-ID: <200902120914.26028.alexandre.fayolle@logilab.fr> Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez ?crit?: > Ahh, yes, now I remember. Psyco ONLY works with 32 bit python, whereas > I MUST use 64 bit because the memory footprint for cvs2svn with our repo > is just too large for 32 bit processes. > > So, dead in the water again? Hmm, just a quick question: are you sure your problem is CPU bound (as opposed to IO bound)? If it is not, you are looking at the wrong place to solve your performance issue, and you should rather look for a faster disk, faster network, etc. Also from what you say hereabove, maybe adding some RAM could help. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 481 bytes Desc: This is a digitally signed message part. URL: From santagada at gmail.com Thu Feb 12 12:20:25 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 12 Feb 2009 09:20:25 -0200 Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm In-Reply-To: <200902120914.26028.alexandre.fayolle@logilab.fr> References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com> <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com> <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com> <200902120914.26028.alexandre.fayolle@logilab.fr> Message-ID: On Feb 12, 2009, at 6:14 AM, Alexandre Fayolle wrote: > Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez > ?crit : >> Ahh, yes, now I remember. Psyco ONLY works with 32 bit python, >> whereas >> I MUST use 64 bit because the memory footprint for cvs2svn with our >> repo >> is just too large for 32 bit processes. >> >> So, dead in the water again? > > Hmm, just a quick question: are you sure your problem is CPU bound > (as opposed > to IO bound)? If it is not, you are looking at the wrong place to > solve your > performance issue, and you should rather look for a faster disk, > faster > network, etc. Also from what you say hereabove, maybe adding some > RAM could > help. I'm answering this as we already talked about it in some private emails. He already figured out the problem is cpu bond, one of the cpus goes to 100% and stays almost the whole time this way for 6.5 days. His machine has 32 gb of RAM and the python conversion program uses good part of it that's why he can't run the thing on 32bit mode. His repo is only 6gb in size so I think maybe the problem is either a bad performing algo no cvs2svn or a bad cvs library (or whatever is used to access cvs). If it is the cvs library there is nothing no one can do (even psyco or pypy would not help). If the problem is a wrong pure python algo it should probably be changed (if a better one exists of course). Now why would a program reading a 6gb repo, even using a database file for intermediate data still uses tons of ram to do this conversion I don't know. -- Leonardo Santagada santagada at gmail.com From stephan at transvection.de Mon Feb 16 19:17:33 2009 From: stephan at transvection.de (Stephan Diehl) Date: Mon, 16 Feb 2009 19:17:33 +0100 Subject: [pypy-dev] [Stackless] Fwd: Stackless on top of greenlets In-Reply-To: <952d92df0902160822q2baf5712jda9de5947a7215e3@mail.gmail.com> References: <7bc21cb60902160210r1de6e235o63006f5d6f2abfbc@mail.gmail.com> <952d92df0902160442p633f8a5fp26b9916cf5aec130@mail.gmail.com> <7bc21cb60902160733s26ec2e4o30153a1797befaa9@mail.gmail.com> <952d92df0902160821oef07395uab0938cf73104033@mail.gmail.com> <952d92df0902160822q2baf5712jda9de5947a7215e3@mail.gmail.com> Message-ID: <4999ADBD.5050301@transvection.de> Hi, it would be quite nice, if the different implementations could somehow be united (I've written the pypy one, but never used it, so there are indeed lots of errors lurking). pypy's stackless is actually implemented using pypy's coroutine implementation, which, in case CPython is used, is emulated by greenlets. pypy's stackless http://codespeak.net/svn/pypy/trunk/pypy/lib/stackless.py has a corresponding test file http://codespeak.net/svn/pypy/trunk/pypy/lib/app_test/test_stackless.py which is supposed to be compatible with regular stackless as well. Maybe we could come up with a set of tests that cover all of stackless interface and therefore can be used to test our greenlet based implementations against. Cheers, Stephan Richard Tew wrote: > ---------- Forwarded message ---------- > From: Richard Tew > Date: Mon, Feb 16, 2009 at 11:21 AM > Subject: Re: [Stackless] Stackless on top of greenlets > To: Henk Punt > > > On Mon, Feb 16, 2009 at 10:33 AM, Henk Punt wrote: >> Can you give some pointers to those implementations?, >> >> The only one I could find is the one included in PyPy and that one is >> not-working, e.g. it lacks certains methods in the API and also it >> does not behave >> exactly as stackless does, especially w.r.t. the order of task-scheduling. >> Also it contains quite a lot of bugs, for instance raising an >> exception in a task that is >> blocked on a channel does not restore the correct balance and similar issues... >> > > None of the implementations I have seen have been perfect, including my own. > > http://www.stackless.com/pipermail/stackless/2005-May/002416.html > > Cheers, > Richard. > > _______________________________________________ > Stackless mailing list > Stackless at stackless.com > http://www.stackless.com/mailman/listinfo/stackless From amauryfa at gmail.com Tue Feb 17 19:20:12 2009 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 17 Feb 2009 19:20:12 +0100 Subject: [pypy-dev] pypy and the comtypes module Message-ID: Hello Thomas, hello to all pypy-ers, I'm sure you will be interested to know that the ctypes implementation of pypy has matured enough to be able to run the comtypes module, at least some of the examples I found on the web site: http://starship.python.net/crew/theller/comtypes/ See the copy of a pypy session below. Not everything works, though. For example, the COMError exception does not give any message, some errors are caused by the alternate garbage collector of pypy (opposed to the determinism of reference counting), and I'm sure that I missed many important details... At least it is now possible to run the comtypes test suite: some tests even pass ;-) Hey, I did not know that ctypes and comtypes where so intricate: ctypes obviously has features designed for comtypes only (OUT parameters...) and ctypes/__init__.py even tries to import comtypes.server.inprocserver! But this was all fun, and I learned a lot about ffi and COM from reading the code. Thanks for these wonderful modules! Python 2.5.2 (61966, Feb 17 2009, 13:39:17) [PyPy 1.0.0] on win32 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``who am I and if yes, how many?'' >>>> from comtypes.client import CreateObject >>>> xl = CreateObject("Excel.Application") # Generating comtypes.gen._00020813_0000_0000_C000_000000000046_0_1_3 # Generating comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0 # Generating comtypes.gen.stdole # Generating comtypes.gen._2DF8D04C_5BFA_101B_BDE5_00AA0044DE52_0_2_1 # Generating comtypes.gen.Office # Generating comtypes.gen._0002E157_0000_0000_C000_000000000046_0_5_3 # Generating comtypes.gen.VBIDE # Generating comtypes.gen.Excel >>>> xl.Workbooks.Add() >>>> xl.Range("E1").Formula = "=SUM(A1:D1)" >>>> print xl.Range("E1").Value 0.0 >>>> xl.Range("A1", "D1").Value = ("Data", 10, 20, 30) >>>> print xl.Range("E1").Value 60.0 >>>> xl.Visible = True >>>> Cheers, -- Amaury Forgeot d'Arc From dalius.dobravolskas at gmail.com Wed Feb 18 14:01:45 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Wed, 18 Feb 2009 15:01:45 +0200 Subject: [pypy-dev] pypy sandbox Message-ID: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> Hello, All, I'm trying to translate sandboxed pypy but I get following error: [sandbox:WARNING] Not Implemented: sandboxing for external function 'inet_ntop' * [rtyper] -=- specialized 3 more blocks -=- [Timer] Timings: [Timer] annotate --- 1380.9 s [Timer] rtype_lltype --- 1414.1 s [Timer] backendopt_lltype --- 647.0 s [Timer] stackcheckinsertion_lltype --- 218.3 s [Timer] database_c --- 1395.4 s [Timer] =========================================== [Timer] Total: --- 5055.7 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 278, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 802, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 270, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 480, in task_database_c [translation:ERROR] database = cbuilder.build_database() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/genc.py", line 73, in build_database [translation:ERROR] db.complete() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 310, in complete [translation:ERROR] add_dependencies(node.enum_dependencies()) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 298, in add_dependencies [translation:ERROR] self.get(value) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 218, in get [translation:ERROR] node = self.getcontainernode(container) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line 156, in getcontainernode [translation:ERROR] node = nodefactory(self, T, container, **buildkwds) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 699, in __init__ [translation:ERROR] self.make_funcgens() [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 705, in make_funcgens [translation:ERROR] self.funcgens = select_function_code_generators(self.obj, self.db, self.name) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 825, in select_function_code_generators [translation:ERROR] return sandbox_stub(fnobj, db) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 787, in sandbox_stub [translation:ERROR] force_stub=True) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/translator/sandbox/rsandbox.py", line 143, in get_external_function_sandbox_graph [translation:ERROR] args_s = [annmodel.lltype_to_annotation(ARG) for ARG in FUNCTYPE.ARGS] [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 639, in lltype_to_annotation [translation:ERROR] return SomePtr(T) [translation:ERROR] File "/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 537, in __init__ [translation:ERROR] assert isinstance(ll_ptrtype, lltype.Ptr) [translation:ERROR] AssertionError [translation] start debugger... > /home/dalius/projects/pypy-dist/pypy/annotation/model.py(537)__init__() -> assert isinstance(ll_ptrtype, lltype.Ptr) (Pdb+) q I was running command: /pypy-dist/pypy/translator/goal$ ./translate.py --sandbox --opt=1 --gc=hybrid targetpypystandalone.py I have just quited this time but it failed with same error as I run ./translate.py --sandbox targetpypystandalone.py (without options). ll_ptrtype is sock_addr if I remember correctly (when I will run it next time I will check). My system: $ uname -a Linux dalius 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009 i686 GNU/Linux $ python -V Python 2.5.2 -- Dalius http://blog.sandbox.lt From anto.cuni at gmail.com Wed Feb 18 14:08:30 2009 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 18 Feb 2009 14:08:30 +0100 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> Message-ID: <499C084E.6040806@gmail.com> Dalius Dobravolskas wrote: > Hello, All, Hi Dalius, [cut] >> /home/dalius/projects/pypy-dist/pypy/annotation/model.py(537)__init__() Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk to dist soon) please try with pypy/trunk. ciao, Anto From dalius.dobravolskas at gmail.com Wed Feb 18 21:39:48 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Wed, 18 Feb 2009 22:39:48 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <499C084E.6040806@gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> Message-ID: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> Hello, On Wed, Feb 18, 2009 at 3:08 PM, Antonio Cuni wrote: > Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk > to dist soon) please try with pypy/trunk. trunk has worked just perfect. I will try to implement WSGI sample similar to Django (as mentioned here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html). -- Dalius http://blog.sandbox.lt From anto.cuni at gmail.com Wed Feb 18 23:45:21 2009 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 18 Feb 2009 23:45:21 +0100 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> Message-ID: <499C8F81.6070103@gmail.com> Dalius Dobravolskas wrote: > trunk has worked just perfect. nice :-) > I will try to implement WSGI sample similar to Django (as mentioned > here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html). that's cool. Please keep us informed, and feel free to come to #pypy on irc.freenode.net if you have any question. ciao, Anto From theller at ctypes.org Thu Feb 19 21:41:48 2009 From: theller at ctypes.org (Thomas Heller) Date: Thu, 19 Feb 2009 21:41:48 +0100 Subject: [pypy-dev] pypy and the comtypes module In-Reply-To: References: Message-ID: <499DC40C.3060801@ctypes.org> [I added comtypes-users to the list of recipients, in case any existing comtypes users is interested in this] Amaury Forgeot d'Arc schrieb: > Hello Thomas, hello to all pypy-ers, > > I'm sure you will be interested to know that the ctypes implementation of pypy > has matured enough to be able to run the comtypes module, > at least some of the examples I found on the web site: > http://starship.python.net/crew/theller/comtypes/ > See the copy of a pypy session below. Really cool to hear this. I must say that I sometimes look over the commit messages in the pypy.cvs mailing list (via gmane) and I'm fascinated by the work with ctypes that is done there! What I fear is that the day is not far where you know much more than myself about ctypes ;-) > Not everything works, though. > For example, the COMError exception does not give any message, > some errors are caused by the alternate garbage collector of pypy > (opposed to the determinism of reference counting), > and I'm sure that I missed many important details... > At least it is now possible to run the comtypes test suite: some tests > even pass ;-) > > Hey, I did not know that ctypes and comtypes where so intricate: > ctypes obviously has features designed for comtypes only (OUT parameters...) > and ctypes/__init__.py even tries to import comtypes.server.inprocserver! > > But this was all fun, and I learned a lot about ffi and COM from > reading the code. > Thanks for these wonderful modules! > > > > Python 2.5.2 (61966, Feb 17 2009, 13:39:17) > [PyPy 1.0.0] on win32 > Type "help", "copyright", "credits" or "license" for more information. > And now for something completely different: ``who am I and if yes, how many?'' >>>>> from comtypes.client import CreateObject >>>>> xl = CreateObject("Excel.Application") > # Generating comtypes.gen._00020813_0000_0000_C000_000000000046_0_1_3 > # Generating comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0 > # Generating comtypes.gen.stdole > # Generating comtypes.gen._2DF8D04C_5BFA_101B_BDE5_00AA0044DE52_0_2_1 > # Generating comtypes.gen.Office > # Generating comtypes.gen._0002E157_0000_0000_C000_000000000046_0_5_3 > # Generating comtypes.gen.VBIDE > # Generating comtypes.gen.Excel >>>>> xl.Workbooks.Add() > >>>>> xl.Range("E1").Formula = "=SUM(A1:D1)" >>>>> print xl.Range("E1").Value > 0.0 >>>>> xl.Range("A1", "D1").Value = ("Data", 10, 20, 30) >>>>> print xl.Range("E1").Value > 60.0 >>>>> xl.Visible = True >>>>> > > Cheers, > Fantastic! I wish I had time to use pypy or even participate, but there is no chance, unfortunately. -- Thanks, Thomas From dalius.dobravolskas at gmail.com Fri Feb 20 07:30:32 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Fri, 20 Feb 2009 08:30:32 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <499C8F81.6070103@gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> Message-ID: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> Hello, >> I will try to implement WSGI sample similar to Django (as mentioned >> here >> http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html). > > that's cool. Please keep us informed, and feel free to come to #pypy on > irc.freenode.net if you have any question. Here is result: http://py.sandbox.lt/ Basically it is the same code as Django example but only written in WSGI (I will share that later). Here some of my experiences that I think I should mention: 1. I'm running my code behind Apache mod_wsgi. As you might know mod_wsgi is very strict WSGI implementation. pypy_interact failed when was calling isatty function (mod_wsgi.Log does not have this function). I have fixed that by adding "'isatty in dir(log)". I will give later exact locations and names if I have confused them. 2. It looks like that PyPySandboxedProc expects --heapsize as first argument in argument list. 3. Is settimeout actually working? If I run following code (please don't run it on my server): x = 1 for y in xrange(1, 1024*1024): x *= y print x Process is not terminated after one second and uses significant part of my CPU resources. I set timeout in following way: sandproc = pypy_interact.PyPySandboxedProc(SANDBOX_BIN, ['--heapsize', str(1024*1024), '-c', code]) sandproc.settimeout(1, interrupt_main=True) I have tried 5 seconds as well. I will analyze the code myself a little bit but if you have ideas already please share them ;-) -- Dalius http://blog.sandbox.lt From dalius.dobravolskas at gmail.com Fri Feb 20 09:05:45 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Fri, 20 Feb 2009 10:05:45 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> Message-ID: <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com> Hello, On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas wrote: > 3. Is settimeout actually working? If I run following code (please > don't run it on my server): OK. Everything is OK in PyPy code. timeout works (at least log indicates that). There is something specific about mod_wsgi and Apache in this case. -- Dalius http://blog.sandbox.lt From dalius.dobravolskas at gmail.com Fri Feb 20 10:13:11 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Fri, 20 Feb 2009 11:13:11 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com> Message-ID: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com> On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas wrote: > Hello, > > On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas > wrote: >> 3. Is settimeout actually working? If I run following code (please >> don't run it on my server): > OK. Everything is OK in PyPy code. timeout works (at least log > indicates that). There is something specific about mod_wsgi and Apache > in this case. That's what I have found out: pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. That's how I solved problem now (modifed pypy/tool/killsubprocess.py). I will dig deeper of course but I will do that later. HTH for other people who will run sandboxed pypy. -- Dalius http://blog.sandbox.lt From fijall at gmail.com Fri Feb 20 13:42:22 2009 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 20 Feb 2009 13:42:22 +0100 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com> <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com> Message-ID: <693bc9ab0902200442t44bb52a6l58bb2945ddeccea1@mail.gmail.com> That's all very good news. Keep us informed! Cheers, fijal On Fri, Feb 20, 2009 at 10:13 AM, Dalius Dobravolskas wrote: > On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas > wrote: >> Hello, >> >> On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas >> wrote: >>> 3. Is settimeout actually working? If I run following code (please >>> don't run it on my server): >> OK. Everything is OK in PyPy code. timeout works (at least log >> indicates that). There is something specific about mod_wsgi and Apache >> in this case. > That's what I have found out: > pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. > That's how I solved problem now (modifed pypy/tool/killsubprocess.py). > I will dig deeper of course but I will do that later. > > HTH for other people who will run sandboxed pypy. > > -- > Dalius > http://blog.sandbox.lt > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From exarkun at divmod.com Fri Feb 20 16:05:51 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Fri, 20 Feb 2009 10:05:51 -0500 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com> Message-ID: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com> On Fri, 20 Feb 2009 11:13:11 +0200, Dalius Dobravolskas wrote: >On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas > wrote: >> Hello, >> >> On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas >> wrote: >>> 3. Is settimeout actually working? If I run following code (please >>> don't run it on my server): >> OK. Everything is OK in PyPy code. timeout works (at least log >> indicates that). There is something specific about mod_wsgi and Apache >> in this case. >That's what I have found out: >pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. >That's how I solved problem now (modifed pypy/tool/killsubprocess.py). >I will dig deeper of course but I will do that later. Check out the sigprocmask Apache sets for itself. Unfortunately, this is inherited by all processes Apache runs, usually leading to surprising behavior, since it prevents the delivery of almost all signals. Jean-Paul From arigo at tunes.org Fri Feb 20 16:59:06 2009 From: arigo at tunes.org (Armin Rigo) Date: Fri, 20 Feb 2009 16:59:06 +0100 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> Message-ID: <20090220155905.GA3445@code0.codespeak.net> Hi Dalius, On Fri, Feb 20, 2009 at 08:30:32AM +0200, Dalius Dobravolskas wrote: > 2. It looks like that PyPySandboxedProc expects --heapsize as first > argument in argument list. Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy translated with the default GC, which is the hybrid GC. I recommend the generational GC (--gc=generation). (This should really warn you...) A bientot, Armin. From holger at merlinux.eu Sat Feb 21 12:39:36 2009 From: holger at merlinux.eu (holger krekel) Date: Sat, 21 Feb 2009 12:39:36 +0100 Subject: [pypy-dev] pytest news: plugins, pycon, release-separation Message-ID: <20090221113936.GQ4576@trillke.net> Hi all, as only some of you are also following pytest developments, find below some news relevant to PyPy as well i think - i plan to merge the new pytest into py/trunk next week which is an svn-external for PyPy. If you have any feedback, i think it's ok if you send it here to pypy-dev, particularly if it is about pypy-related concerns. cheers, holger Hi folks, some good news. as those glimpsing at pytest svn-commits know i am heavily working on a new plugin architecture. I am really happy about it as those also following me on my brand-new twitter account (hpk42) might guess. Around Pycon i am giving a tutorial about advanced testing topics including cross-platform distributed testing and writing plugins and i plan to talk in detail about the new and much improved ways of doing things. Here is the status: * Terminal reporting is now a nice self-contained plugin * there is a new "xfail" plugin, inspired by SimPy to mark test functions as "expected to fail" and report specially * a "poccoo" plugin for sending test failures to http://paste.pocoo.org * new plugins for managing setup/teardown of application state, using a turns-out-to-work-great new mechanism which i guess nosetests might imitate at some point :) Regarding the next release i consider doing the following: * open two google-code projects "pytest" and "pylib" with their svn repos remaining at codespeak (where many projects share one respository - i prefer that) * move current py/test/ to its own "pytest/" root project and depend on the py lib for install. * aim to do two alpha releases, pytest-1.0 alpha and py lib 1.0 alpha (containing the rest) What do you think? I am unfamiliar with commoditized code hosting and consider using google code. I'd probably like to also have an issue tracker for pytest. Any comments/recommendations? Would someone be up for co-admining such a project? Meanwhile i am finishing up the pytestplugins branch for merge to trunk, stay tuned :) cheers & have fun, hope to meet some of you at Pycon, holger -- collaborative expert contracting: http://merlinux.eu PyPy Python/Compiler tool chain: http://codespeak.net/pypy pylib py.test/greenlets/svn APIs: http://pylib.org _______________________________________________ py-dev mailing list py-dev at codespeak.net http://codespeak.net/mailman/listinfo/py-dev ----- End forwarded message ----- -- collaborative expert contracting: http://merlinux.eu PyPy Python/Compiler tool chain: http://codespeak.net/pypy pylib py.test/greenlets/svn APIs: http://pylib.org From holger at merlinux.eu Sun Feb 22 17:40:32 2009 From: holger at merlinux.eu (holger krekel) Date: Sun, 22 Feb 2009 17:40:32 +0100 Subject: [pypy-dev] new way to organise test state / new blog Message-ID: <20090222164032.GU4576@trillke.net> Hi py-dev, testing-in-python and pypy-dev, i just created a blog where i intend to write about news regarding testing and python and metaprogramming. http://tetamap.blogspot.com/ and did a first posting about a new way of doing test state setup with py.test. Um, does anybody know how to do code coloring with blogspot? cheers, holger -- collaborative expert contracting: http://merlinux.eu PyPy Python/Compiler tool chain: http://codespeak.net/pypy pylib py.test/greenlets/svn APIs: http://pylib.org From jgustak at gmail.com Sun Feb 22 18:08:16 2009 From: jgustak at gmail.com (Jakub Gustak) Date: Sun, 22 Feb 2009 18:08:16 +0100 Subject: [pypy-dev] new way to organise test state / new blog In-Reply-To: <20090222164032.GU4576@trillke.net> References: <20090222164032.GU4576@trillke.net> Message-ID: > i just created a blog where i intend to write about news > regarding testing and python and metaprogramming. > > http://tetamap.blogspot.com/ Next blog to subscribe. Good. > and did a first posting about a new way of doing test state > setup with py.test. Um, does anybody know how to do code coloring > with blogspot? You might want to try this out: http://techqi.blogspot.com/2009/02/how-to-display-code-nicely-in-blogger.html http://alexgorbatchev.com/wiki/SyntaxHighlighter Cheers, Jakub From holger at merlinux.eu Sun Feb 22 19:09:51 2009 From: holger at merlinux.eu (holger krekel) Date: Sun, 22 Feb 2009 19:09:51 +0100 Subject: [pypy-dev] new way to organise test state / new blog In-Reply-To: <20090222164032.GU4576@trillke.net> References: <20090222164032.GU4576@trillke.net> Message-ID: <20090222180951.GV4576@trillke.net> Hello again, thanks to all! I'll check out the suggested code coloring options. note that i decided to move to wordpress already, i.e. http://tetamap.wordpress.com/ because i like the interface possibilities there better. sorry for the noise. please send any more comments to me privately and not as a cross-post. best, holger On Sun, Feb 22, 2009 at 17:40 +0100, holger krekel wrote: > Hi py-dev, testing-in-python and pypy-dev, > > i just created a blog where i intend to write about news > regarding testing and python and metaprogramming. > > http://tetamap.blogspot.com/ > > and did a first posting about a new way of doing test state > setup with py.test. Um, does anybody know how to do code coloring > with blogspot? > > cheers, > > holger > > -- > collaborative expert contracting: http://merlinux.eu > PyPy Python/Compiler tool chain: http://codespeak.net/pypy > pylib py.test/greenlets/svn APIs: http://pylib.org > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- collaborative expert contracting: http://merlinux.eu PyPy Python/Compiler tool chain: http://codespeak.net/pypy pylib py.test/greenlets/svn APIs: http://pylib.org From donny.viszneki at gmail.com Sun Feb 22 19:34:42 2009 From: donny.viszneki at gmail.com (Donny Viszneki) Date: Sun, 22 Feb 2009 13:34:42 -0500 Subject: [pypy-dev] new way to organise test state / new blog In-Reply-To: <20090222164032.GU4576@trillke.net> References: <20090222164032.GU4576@trillke.net> Message-ID: <44acbb800902221034o2979c390p5ce1dc3205534fb9@mail.gmail.com> On Sun, Feb 22, 2009 at 11:40 AM, holger krekel wrote: > Um, does anybody know how to do code coloring > with blogspot? If you use vim, it's :TOhtml -- http://codebad.com/ From dalius.dobravolskas at gmail.com Sun Feb 22 20:46:58 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Sun, 22 Feb 2009 21:46:58 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com> References: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com> Message-ID: <49A1ABB2.3090006@gmail.com> Jean-Paul Calderone wrote: >> That's what I have found out: >> pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed. >> That's how I solved problem now (modifed pypy/tool/killsubprocess.py). >> I will dig deeper of course but I will do that later. >> > > Check out the sigprocmask Apache sets for itself. Unfortunately, this > is inherited by all processes Apache runs, usually leading to surprising > behavior, since it prevents the delivery of almost all signals. > I fear that everything is more complicated here. There are some information in mod_wsgi documentation about signals. I will ask Graham (mod_wsgi author) about that. He might have some ideas what to do. Regards, Dalius From dalius.dobravolskas at gmail.com Sun Feb 22 20:57:22 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Sun, 22 Feb 2009 21:57:22 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> Message-ID: <49A1AE22.2090002@gmail.com> Hello, Dalius Dobravolskas wrote: > 1. I'm running my code behind Apache mod_wsgi. As you might know > mod_wsgi is very strict WSGI implementation. pypy_interact failed when > was calling isatty function (mod_wsgi.Log does not have this > function). I have fixed that by adding "'isatty in dir(log)". I will > give later exact locations and names if I have confused them Here is traceback from this situation: Traceback (most recent call last):, referer: http://py.sandbox.lt/ File "/home/dalius/wsgi/pypysandboxserver.py", line 25, in __call__, referer: http://py.sandbox.lt/ sandproc.interact(stdout=code_output, stderr=code_output), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 307, in interact, referer: http://py.sandbox.lt/ returncode = self.handle_until_return(), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", line 228, in handle_until_return, referer: http://py.sandbox.lt/ log.call('%s(%s)' % (fnname,, referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/log/producer.py", line 55, in __call__, referer: http://py.sandbox.lt/ func(self.Message(self.keywords, args)), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/pypy/tool/ansi_print.py", line 72, in __call__, referer: http://py.sandbox.lt/ file=self.file, newline=newline, flush=flush), referer: http://py.sandbox.lt/ File "/home/dalius/projects/pypy-trunk/py/io/terminalwriter.py", line 34, in ansi_print, referer: http://py.sandbox.lt/ if esc and sys.platform != "win32" and file.isatty():, referer: http://py.sandbox.lt/ AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty', referer: http://py.sandbox.lt/ Here is diff how I have avoided this problem: --- io/terminalwriter.py (revision 62015) +++ io/terminalwriter.py (working copy) @@ -31,7 +31,7 @@ if file is None: file = sys.stderr text = text.rstrip() - if esc and sys.platform != "win32" and file.isatty(): + if esc and sys.platform != "win32" and 'isatty' in dir(file) and file.isatty(): if not isinstance(esc, tuple): esc = (esc,) text = (''.join(['\x1b[%sm' % cod for cod in esc]) + Could it be fixed in the code? Regards, Dalius From cfbolz at gmx.de Mon Feb 23 16:17:36 2009 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 23 Feb 2009 16:17:36 +0100 Subject: [pypy-dev] JavaScript parser in Python Message-ID: <49A2BE10.1080609@gmx.de> Hi all, in case we ever get more serious about our JS interpreter, there seems to be a pure-Python port of the JavaScript parser of the Narcissus-project here: http://code.google.com/p/pynarcissus/ Automatic semicolon insertion included. Cheers, Carl Friedrich From santagada at gmail.com Tue Feb 24 01:05:20 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 23 Feb 2009 21:05:20 -0300 Subject: [pypy-dev] JavaScript parser in Python In-Reply-To: <49A2BE10.1080609@gmx.de> References: <49A2BE10.1080609@gmx.de> Message-ID: On Feb 23, 2009, at 12:17 PM, Carl Friedrich Bolz wrote: > Hi all, > > in case we ever get more serious about our JS interpreter, there seems > to be a pure-Python port of the JavaScript parser of the > Narcissus-project here: > > http://code.google.com/p/pynarcissus/ > > Automatic semicolon insertion included. There was another port of the narcissus parser, the problem being that the license is incompatible with the mit license. Another idea is that the webkit parser is made using byson, if we had a yacc or something like that it would be trivial to port, and I think the license is more permissive (or I just forgot about it and it is completely incompatible with the mit license). -- Leonardo Santagada santagada at gmail.com From marius at pov.lt Tue Feb 24 17:18:33 2009 From: marius at pov.lt (Marius Gedminas) Date: Tue, 24 Feb 2009 18:18:33 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <49A1AE22.2090002@gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> <49A1AE22.2090002@gmail.com> Message-ID: <20090224161833.GB21714@fridge.pov.lt> On Sun, Feb 22, 2009 at 09:57:22PM +0200, Dalius Dobravolskas wrote: > Here is diff how I have avoided this problem: > > --- io/terminalwriter.py (revision 62015) > +++ io/terminalwriter.py (working copy) > @@ -31,7 +31,7 @@ > if file is None: > file = sys.stderr > text = text.rstrip() > - if esc and sys.platform != "win32" and file.isatty(): > + if esc and sys.platform != "win32" and 'isatty' in dir(file) and > file.isatty(): Please use hasattr(log, 'isatty') instead. > if not isinstance(esc, tuple): > esc = (esc,) > text = (''.join(['\x1b[%sm' % cod for cod in esc]) + Marius Gedminas -- I am a computer. I am dumber than any human and smarter than any administrator. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From dalius.dobravolskas at gmail.com Wed Feb 25 06:43:19 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Wed, 25 Feb 2009 07:43:19 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <20090224161833.GB21714@fridge.pov.lt> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> <49A1AE22.2090002@gmail.com> <20090224161833.GB21714@fridge.pov.lt> Message-ID: <98dd087e0902242143q2257a008t6f9d6ec92b808faa@mail.gmail.com> >> + ? ?if esc and sys.platform != "win32" and 'isatty' in dir(file) and >> file.isatty(): > > Please use hasattr(log, 'isatty') instead. A?i?, Mariau. I felt that there should be more efficient way to do that ;-) -- Dalius http://blog.sandbox.lt From dalius.dobravolskas at gmail.com Wed Feb 25 06:49:36 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Wed, 25 Feb 2009 07:49:36 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com> References: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com> <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com> Message-ID: <98dd087e0902242149h320d8544sf862578ac1f9ebb@mail.gmail.com> > Check out the sigprocmask Apache sets for itself. ?Unfortunately, this > is inherited by all processes Apache runs, usually leading to surprising > behavior, since it prevents the delivery of almost all signals. FYI, I have contacted mod_wsgi author. If you are using mod_wsgi 2.X with deamon mode SIGTERM works. The problem still persists for embedded mode. That means there is solution that works with SIGTERM and no changes are required in pypy code. If you are interested how it will work in the future, please follow this thread in modwsgi group: http://groups.google.com/group/modwsgi/browse_thread/thread/aae8d2d675ebcf9 Thanks everyone for your help. -- Dalius http://blog.sandbox.lt From arigo at tunes.org Wed Feb 25 19:12:21 2009 From: arigo at tunes.org (Armin Rigo) Date: Wed, 25 Feb 2009 19:12:21 +0100 Subject: [pypy-dev] pypy sandbox In-Reply-To: <20090220155905.GA3445@code0.codespeak.net> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> <499C8F81.6070103@gmail.com> <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com> <20090220155905.GA3445@code0.codespeak.net> Message-ID: <20090225181221.GA19288@code0.codespeak.net> Hi, On Fri, Feb 20, 2009 at 04:59:06PM +0100, Armin Rigo wrote: > Bit of a hack, but yes, indeed. It also doesn't work well with a PyPy > translated with the default GC, which is the hybrid GC. I recommend the > generational GC (--gc=generation). (This should really warn you...) Ah, I just found a hack in pypy/config/translationoption.py. The GC defaults to 'generation' instead of 'hybrid' in case of a sandbox translation. So everything is fine :-) A bientot, Armin. From santagada at gmail.com Thu Feb 26 15:08:51 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 26 Feb 2009 11:08:51 -0300 Subject: [pypy-dev] Parsing in PyPy (and runicode) Message-ID: I remember some time ago people on #pypy were talking about redoing the parser for python because it was not good enough or something. Also the parser generator that cfbolz wrote doesn't support unicode and was not suited for automatic-semicolon-insertion. I think it would be good for a javascript parser that supports unicode because the specs call for it and maybe it would be good to python too (don't know about prolog/smalltalk though). What would be better, to have a parser generator that supports unicode or just everyone write their own recursive descendant parser by hand? -- Leonardo Santagada santagada at gmail.com From cfbolz at gmx.de Thu Feb 26 18:40:08 2009 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 26 Feb 2009 18:40:08 +0100 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: References: Message-ID: <49A6D3F8.3020501@gmx.de> Hi Leonardo, Leonardo Santagada wrote: > I remember some time ago people on #pypy were talking about redoing > the parser for python because it was not good enough or something. > Also the parser generator that cfbolz wrote doesn't support unicode > and was not suited for automatic-semicolon-insertion. > > I think it would be good for a javascript parser that supports unicode > because the specs call for it and maybe it would be good to python too > (don't know about prolog/smalltalk though). What would be better, to > have a parser generator that supports unicode or just everyone write > their own recursive descendant parser by hand? Are you sure that we shouldn't rather try to steal one of the existing JS parsers? If license really is a problem, maybe we could ask the authors whether they would be fine with relicensing under MIT. Cheers, Carl Friedrich From santagada at gmail.com Thu Feb 26 19:29:15 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 26 Feb 2009 15:29:15 -0300 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <49A6D3F8.3020501@gmx.de> References: <49A6D3F8.3020501@gmx.de> Message-ID: <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> On Feb 26, 2009, at 2:40 PM, Carl Friedrich Bolz wrote: > Hi Leonardo, > > Leonardo Santagada wrote: >> I remember some time ago people on #pypy were talking about redoing >> the parser for python because it was not good enough or something. >> Also the parser generator that cfbolz wrote doesn't support unicode >> and was not suited for automatic-semicolon-insertion. >> >> I think it would be good for a javascript parser that supports >> unicode >> because the specs call for it and maybe it would be good to python >> too >> (don't know about prolog/smalltalk though). What would be better, to >> have a parser generator that supports unicode or just everyone write >> their own recursive descendant parser by hand? > > Are you sure that we shouldn't rather try to steal one of the existing > JS parsers? If license really is a problem, maybe we could ask the > authors whether they would be fine with relicensing under MIT. All the ones that come from narcissus are actually just licensed by Mozilla, so we should talk to them. But I don't know if it is complete and without bugs (I think there was one with narcissus but I don't remember what was it) so maybe we should use one used by a functioning js interpreters like Rhino of JavascriptCore (Squirrelfish Extreme or whatever its name is now). Both uses an incompatible license, and I think it is on purpose... so I don't know how to deal with this. The V8 parser is BSD so I think it is compatible, but it will be some work to convert it from c++ to rpython (http://code.google.com/p/v8/source/browse/trunk/src/parser.cc ) My final answer would be, "I don't know, what do you guys think?". -- Leonardo Santagada santagada at gmail.com From jbaker at zyasoft.com Thu Feb 26 19:44:07 2009 From: jbaker at zyasoft.com (Jim Baker) Date: Thu, 26 Feb 2009 11:44:07 -0700 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> Message-ID: Perhaps consider Antlr? We've had good success with this for Jython, and it's now also used by NetBeans support for Python in general. I took a look at http://www.antlr.org/grammar/list, and there are a number of options for JS. Most importantly, Antlr supports the parser chain in Python, so it's possible this could be more readily converted to RPython. Some potential issues: Chris Lambrou has a parser for EcmaScript 3.0. But there's no license here, so you'd definitely have to contact him on this. Like standalone grammars in general, it's rather unlikely to have been extensively tested. With Jython we started with a reference parser that Terrence Parr had made, and taking CPython as reference here as to what the AST should be, over time we targeted that by explicitly comparing ASTS. Some time later, including incremental parse support and various syntax errors, we're pretty confident that it's basically done. - Jim On Thu, Feb 26, 2009 at 11:29 AM, Leonardo Santagada wrote: > > On Feb 26, 2009, at 2:40 PM, Carl Friedrich Bolz wrote: > > > Hi Leonardo, > > > > Leonardo Santagada wrote: > >> I remember some time ago people on #pypy were talking about redoing > >> the parser for python because it was not good enough or something. > >> Also the parser generator that cfbolz wrote doesn't support unicode > >> and was not suited for automatic-semicolon-insertion. > >> > >> I think it would be good for a javascript parser that supports > >> unicode > >> because the specs call for it and maybe it would be good to python > >> too > >> (don't know about prolog/smalltalk though). What would be better, to > >> have a parser generator that supports unicode or just everyone write > >> their own recursive descendant parser by hand? > > > > Are you sure that we shouldn't rather try to steal one of the existing > > JS parsers? If license really is a problem, maybe we could ask the > > authors whether they would be fine with relicensing under MIT. > > > All the ones that come from narcissus are actually just licensed by > Mozilla, so we should talk to them. But I don't know if it is complete > and without bugs (I think there was one with narcissus but I don't > remember what was it) so maybe we should use one used by a functioning > js interpreters like Rhino of JavascriptCore (Squirrelfish Extreme or > whatever its name is now). Both uses an incompatible license, and I > think it is on purpose... so I don't know how to deal with this. > > The V8 parser is BSD so I think it is compatible, but it will be some > work to convert it from c++ to rpython ( > http://code.google.com/p/v8/source/browse/trunk/src/parser.cc > ) > > My final answer would be, "I don't know, what do you guys think?". > > -- > Leonardo Santagada > santagada at gmail.com > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- Jim Baker jbaker at zyasoft.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From santagada at gmail.com Thu Feb 26 20:55:11 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 26 Feb 2009 16:55:11 -0300 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> Message-ID: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> On Feb 26, 2009, at 3:44 PM, Jim Baker wrote: > Perhaps consider Antlr? We've had good success with this for Jython, > and it's now also used by NetBeans support for Python in general. I > took a look at http://www.antlr.org/grammar/list, and there are a > number of options for JS. Most importantly, Antlr supports the > parser chain in Python, so it's possible this could be more readily > converted to RPython. Yep this could be done, if so I would use this grammar file http://research.xebic.com/es3/ and then put the code for JS 1.5+ in it as we start supporting those features. > Some potential issues: > Chris Lambrou has a parser for EcmaScript 3.0. But there's no > license here, so you'd definitely have to contact him on this. Like > standalone grammars in general, it's rather unlikely to have been > extensively tested. With Jython we started with a reference parser > that Terrence Parr had made, and taking CPython as reference here as > to what the AST should be, over time we targeted that by explicitly > comparing ASTS. Some time later, including incremental parse support > and various syntax errors, we're pretty confident that it's > basically done. The thing that would be great is if pypy and jython would use the same parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc) could be partially shared :) -- Leonardo Santagada santagada at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbaker at zyasoft.com Thu Feb 26 21:18:53 2009 From: jbaker at zyasoft.com (Jim Baker) Date: Thu, 26 Feb 2009 13:18:53 -0700 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> Message-ID: Please feel free to use our Python parser, it's licensed under the original BSD license from Terrence Parr. https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython/grammar/ I don't know where the 3.0 work is currently located, in any event it's something we have put on hold for the moment as we get the 2.5 release out. I cc-ed in Frank to ensure he is in the loop here. In terms of the parser itself: there's a modest amount of Java code in the actions, but that should be easy to convert. Supporting RPython generation in Antlr then makes even more sense in this case: as I understand it, Antlr uses TP's other project, StringTemplate, to simplify the construction of multiple backends. - Jim On Thu, Feb 26, 2009 at 12:55 PM, Leonardo Santagada wrote: > > On Feb 26, 2009, at 3:44 PM, Jim Baker wrote: > > Perhaps consider Antlr? We've had good success with this for Jython, and > it's now also used by NetBeans support for Python in general. I took a look > at http://www.antlr.org/grammar/list, and there are a number of options > for JS. Most importantly, Antlr supports the parser chain in Python, so it's > possible this could be more readily converted to RPython. > > > Yep this could be done, if so I would use this grammar file > http://research.xebic.com/es3/ and then put the code for JS 1.5+ in it as > we start supporting those features. > > > Some potential issues: > Chris Lambrou has a parser for EcmaScript 3.0. But there's no license here, > so you'd definitely have to contact him on this. Like standalone grammars in > general, it's rather unlikely to have been extensively tested. With Jython > we started with a reference parser that Terrence Parr had made, and taking > CPython as reference here as to what the AST should be, over time we > targeted that by explicitly comparing ASTS. Some time later, including > incremental parse support and various syntax errors, we're pretty confident > that it's basically done. > > > The thing that would be great is if pypy and jython would use the same > parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc) > could be partially shared :) > > -- > Leonardo Santagada > santagada at gmail.com > > > > -- Jim Baker jbaker at zyasoft.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalius.dobravolskas at gmail.com Fri Feb 27 09:15:25 2009 From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas) Date: Fri, 27 Feb 2009 10:15:25 +0200 Subject: [pypy-dev] pypy sandbox In-Reply-To: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com> <499C084E.6040806@gmail.com> <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com> Message-ID: <98dd087e0902270015q3d2b75fbq992308f3e0da032@mail.gmail.com> Hello, On Wed, Feb 18, 2009 at 10:39 PM, Dalius Dobravolskas wrote: > I will try to implement WSGI sample similar to Django (as mentioned > here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html). So here is final result: http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox Thank you everyone for your help. I think I will play around more with PyPy. -- Dalius http://blog.sandbox.lt From holger at merlinux.eu Fri Feb 27 12:45:20 2009 From: holger at merlinux.eu (holger krekel) Date: Fri, 27 Feb 2009 12:45:20 +0100 Subject: [pypy-dev] pypy/doc changes / py.test plugins Message-ID: <20090227114520.GC4576@trillke.net> Hi all, i just merged the pytest plugin branch to py/trunk which is used by pypy/trunk and branches. I needed to fix pypy/trunk/doc/* to work with the new pytest_restdoc plugin, otherwise i hope that everythings just continues to work. But to avoid deprecation warnings you might want to apply revision 62212 and 62219 to the branches. If you are interested, you might read up on pytest plugins here: http://tetamap.wordpress.com/2009/02/27/new-plugin-architecture-and-plugins-for-pytest/ cheers, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu From fwierzbicki at gmail.com Fri Feb 27 15:42:06 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Fri, 27 Feb 2009 09:42:06 -0500 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> Message-ID: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada wrote: > The thing that would be great is if pypy and jython would use the same > parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc) > could be partially shared :) This would indeed be very cool. Also, ANTLR 3.x supports a really interesting form of grammar inheritance which would help us share a base grammar (I remember it has a sort of diff-merge form of inheritance, my google skills are failing me, I'll find a reference today sometime I'm sure). At the JVM Language summit last year, I met ANTLR expert Jim Idle, and he expressed an interest in seeing if the Jython grammar could be used as a grammar for CPython. I've copied Jim Idle on this email. As a side note, it appears that Guido van Rossum has had some positive experiences with ANTLR recently: http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html -Frank From fwierzbicki at gmail.com Fri Feb 27 15:46:41 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Fri, 27 Feb 2009 09:46:41 -0500 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> Message-ID: <4dab5f760902270646r50c8e72id54b039ab5ac12de@mail.gmail.com> On Thu, Feb 26, 2009 at 3:18 PM, Jim Baker wrote: > I don't know where the 3.0 work is currently located, in any event it's > something we have put on hold for the moment as we get the 2.5 release out. > I cc-ed in Frank to ensure he is in the loop here. The 3.0 grammar is here: https://jython.svn.sourceforge.net/svnroot/jython/branches/jy3k/grammar Some light testing with AST comparisons shows that it is pretty good, but does not yet support the extended function syntax (like keyword only args) -Frank From fwierzbicki at gmail.com Fri Feb 27 15:59:31 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Fri, 27 Feb 2009 09:59:31 -0500 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> References: <49A6D3F8.3020501@gmx.de> <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com> <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> Message-ID: <4dab5f760902270659w5448a729nd768d33710143bd7@mail.gmail.com> On Fri, Feb 27, 2009 at 9:42 AM, Frank Wierzbicki wrote: > (I remember it has a sort of diff-merge form of > inheritance, my google skills are failing me, I'll find a reference > today sometime I'm sure). I found a paper (http://www.cs.vu.nl/icpc2008/docs/Parr.pdf) but I'm not sure if this has made it into official 3.x yet. -Frank From jacob at openend.se Fri Feb 27 19:19:03 2009 From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Fri, 27 Feb 2009 19:19:03 +0100 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> Message-ID: <200902271919.03717.jacob@openend.se> fredagen den 27 februari 2009 skrev Frank Wierzbicki: > On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada wrote: > > The thing that would be great is if pypy and jython would use the same > > parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc) > > could be partially shared :) > > This would indeed be very cool. Also, ANTLR 3.x supports a really > interesting form of grammar inheritance which would help us share a > base grammar (I remember it has a sort of diff-merge form of > inheritance, my google skills are failing me, I'll find a reference > today sometime I'm sure). At the JVM Language summit last year, I met > ANTLR expert Jim Idle, and he expressed an interest in seeing if the > Jython grammar could be used as a grammar for CPython. I've copied > Jim Idle on this email. > > As a side note, it appears that Guido van Rossum has had some positive > experiences with ANTLR recently: > > http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html > > -Frank > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev Andrew Dalke, who is very thorough in his investigation of software, has written som interesting things about his experience with ANTLR as well as some other parsing projects. In short, he likes ANTLR as a tool, but in his application, it is considerably slower than some other alternatives. He also has something called python4ply, which is a ready, MIT licensed parser for Python. You can find his articles on http://www.dalkescientific.com/writings/diary/archive/ Jacob Hall?n From santagada at gmail.com Fri Feb 27 19:37:10 2009 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 27 Feb 2009 15:37:10 -0300 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <200902271919.03717.jacob@openend.se> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> <200902271919.03717.jacob@openend.se> Message-ID: <693E7D78-65B6-4680-A34D-E4999915B172@gmail.com> On Feb 27, 2009, at 3:19 PM, Jacob Hall?n wrote: > fredagen den 27 februari 2009 skrev Frank Wierzbicki: >> On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada > > > wrote: >>> The thing that would be great is if pypy and jython would use the >>> same >>> parser using antlr so the work to support python 3.0 (and 2.7, >>> 2.8, etc) >>> could be partially shared :) >> >> This would indeed be very cool. Also, ANTLR 3.x supports a really >> interesting form of grammar inheritance which would help us share a >> base grammar (I remember it has a sort of diff-merge form of >> inheritance, my google skills are failing me, I'll find a reference >> today sometime I'm sure). At the JVM Language summit last year, I >> met >> ANTLR expert Jim Idle, and he expressed an interest in seeing if the >> Jython grammar could be used as a grammar for CPython. I've copied >> Jim Idle on this email. >> >> As a side note, it appears that Guido van Rossum has had some >> positive >> experiences with ANTLR recently: >> >> http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html >> >> -Frank >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev > > Andrew Dalke, who is very thorough in his investigation of software, > has > written som interesting things about his experience with ANTLR as > well as > some other parsing projects. In short, he likes ANTLR as a tool, but > in his > application, it is considerably slower than some other alternatives. > He also has something called python4ply, which is a ready, MIT > licensed > parser for Python. > > You can find his articles on > http://www.dalkescientific.com/writings/diary/archive/ The problem he might be having is with the python backend for ANTLR, wich neither us (we are going to have to create a rpython one) nor cpython (which would use a c89 one) would have. but this is just a guess as I have had no time to read his article yet -- Leonardo Santagada santagada at gmail.com From fwierzbicki at gmail.com Fri Feb 27 19:38:39 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Fri, 27 Feb 2009 13:38:39 -0500 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <200902271919.03717.jacob@openend.se> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> <200902271919.03717.jacob@openend.se> Message-ID: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com> On Fri, Feb 27, 2009 at 1:19 PM, Jacob Hall?n wrote: > Andrew Dalke, who is very thorough in his investigation of software, has > written som interesting things about his experience with ANTLR as well as > some other parsing projects. In short, he likes ANTLR as a tool, but in his > application, it is considerably slower than some other alternatives. > He also has something called python4ply, which is a ready, MIT licensed > parser for Python. > > You can find his articles on > http://www.dalkescientific.com/writings/diary/archive/ I see the part you are likely talking about: """ It looks like every character read incurs several Python function calls, which are a lot more expensive in Python than in Java or C++. There's no easy change for this, so I'm pretty sure the ANTLR-generated parser always going to be slower than PLY. """ I wonder if producing a parser in C would work for PyPy if this is unavoidable? I know it misses the purpose of PyPy a bit -- just a thought :) -Frank From fwierzbicki at gmail.com Fri Feb 27 19:40:45 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Fri, 27 Feb 2009 13:40:45 -0500 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <693E7D78-65B6-4680-A34D-E4999915B172@gmail.com> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> <200902271919.03717.jacob@openend.se> <693E7D78-65B6-4680-A34D-E4999915B172@gmail.com> Message-ID: <4dab5f760902271040o410aa987te0e62fd3f561e44b@mail.gmail.com> On Fri, Feb 27, 2009 at 1:37 PM, Leonardo Santagada wrote: > The problem he might be having is with the python backend for ANTLR, wich > neither us (we are going to have to create a rpython one) nor cpython (which > would use a c89 one) would have. Ah! You are thinking of generating an rpython backend -- interesting. -Frank From arigo at tunes.org Fri Feb 27 20:19:46 2009 From: arigo at tunes.org (Armin Rigo) Date: Fri, 27 Feb 2009 20:19:46 +0100 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> <200902271919.03717.jacob@openend.se> <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com> Message-ID: <20090227191945.GA4243@code0.codespeak.net> Hi, On Fri, Feb 27, 2009 at 01:38:39PM -0500, Frank Wierzbicki wrote: > """ > It looks like every character read incurs several Python function > calls, which are a lot more expensive in Python than in Java or C++. > There's no easy change for this, so I'm pretty sure the > ANTLR-generated parser always going to be slower than PLY. > """ > > I wonder if producing a parser in C would work for PyPy if this is > unavoidable? I know it misses the purpose of PyPy a bit -- just a > thought :) No, our goal would be to generate an RPython parser, not just a Python one. A few function calls are not more of a problem for RPython than they are for C. So that should be fine. In other words, if ANTLR generates parsers expecting the speed of a Java or C++ kind of backend, then an RPython backend is no problem either. A bientot, Armin. From holger at merlinux.eu Fri Feb 27 20:19:08 2009 From: holger at merlinux.eu (holger krekel) Date: Fri, 27 Feb 2009 20:19:08 +0100 Subject: [pypy-dev] Parsing in PyPy (and runicode) In-Reply-To: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com> References: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com> <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com> <200902271919.03717.jacob@openend.se> <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com> Message-ID: <20090227191908.GL4576@trillke.net> On Fri, Feb 27, 2009 at 13:38 -0500, Frank Wierzbicki wrote: > On Fri, Feb 27, 2009 at 1:19 PM, Jacob Hall?n wrote: > > Andrew Dalke, who is very thorough in his investigation of software, has > > written som interesting things about his experience with ANTLR as well as > > some other parsing projects. In short, he likes ANTLR as a tool, but in his > > application, it is considerably slower than some other alternatives. > > He also has something called python4ply, which is a ready, MIT licensed > > parser for Python. > > > > You can find his articles on > > http://www.dalkescientific.com/writings/diary/archive/ > I see the part you are likely talking about: > > """ > It looks like every character read incurs several Python function > calls, which are a lot more expensive in Python than in Java or C++. > There's no easy change for this, so I'm pretty sure the > ANTLR-generated parser always going to be slower than PLY. > """ > > I wonder if producing a parser in C would work for PyPy if this is > unavoidable? I know it misses the purpose of PyPy a bit -- just a > thought :) RPython calls are not much more expensive than C calls. So an RPython backend would not suffer. holger From jurgis.pralgauskis at gmail.com Sat Feb 28 09:09:20 2009 From: jurgis.pralgauskis at gmail.com (Jurgis Pralgauskis) Date: Sat, 28 Feb 2009 10:09:20 +0200 Subject: [pypy-dev] pypy translation error :\ Message-ID: <34f4097d0902280009p17e54432ked63e3542b271539@mail.gmail.com> Hello, maybe smb could give a hint, why: http://files.akl.lt/users/jurgis/etc/pypy_translation_error.txt ps.: I acted according to instructions on http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox my env: python 2.5.2, Ubuntu 8.04 Thanks in advance -- Jurgis Pralgauskis Don't worry, be happy and make things better ;) http://sagemath.visiems.lt From anto.cuni at gmail.com Sat Feb 28 19:28:31 2009 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sat, 28 Feb 2009 19:28:31 +0100 Subject: [pypy-dev] holydays Message-ID: <49A9824F.20706@gmail.com> Hi all, I'm going on holydays next week (skiing :-)), so I won't be online until the 9th. ciao, Anto From nytrokiss at gmail.com Sat Feb 28 20:03:37 2009 From: nytrokiss at gmail.com (James Matthews) Date: Sat, 28 Feb 2009 21:03:37 +0200 Subject: [pypy-dev] holydays In-Reply-To: <49A9824F.20706@gmail.com> References: <49A9824F.20706@gmail.com> Message-ID: <8a6b8e350902281103r76424ddfv50f7707a95a7b023@mail.gmail.com> Holidays or Holy Days :)? Enjoy your vacation! On Sat, Feb 28, 2009 at 8:28 PM, Antonio Cuni wrote: > Hi all, > I'm going on holydays next week (skiing :-)), so I won't be online until > the 9th. > > ciao, > Anto > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- http://www.astorandblack.com -------------- next part -------------- An HTML attachment was scrubbed... URL: