From arigo at tunes.org Wed Feb 1 13:16:21 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 1 Feb 2006 13:16:21 +0100 Subject: [pypy-dev] Next pypy-sync meeting Message-ID: <20060201121621.GA19407@code0.codespeak.net> Hi all, I'm happy to announce tomorrow's pypy-sync meeting, as usual, at 1pm on #pypy-sync on irc.freenode.net. Topics: * everybody's three-liners (last, next, blockers) * finalize the PyCon sprint announcement -- in particular, the sprint topics * py.test: Holger would like to ask us what we think should be better for running PyPy tests. For example, I think that py.test usually dumps far too much information with PyPy, and it's tedious to have to dig for the most important stuff -- e.g. the actual exception is usually lost between a long traceback and a long stdout dump. Please come up with complains and optionally with ideas on how to fix them :-) A bientot, Armin From arigo at tunes.org Fri Feb 10 15:09:33 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 10 Feb 2006 15:09:33 +0100 Subject: [pypy-dev] PyCon 2006 sprint Message-ID: <20060210140933.GA30708@code0.codespeak.net> Post-PyCon PyPy Sprint: February 27th - March 2nd 2006 ============================================================ The next PyPy sprint is scheduled to take place right after PyCon 2006 in Dallas, Texas, USA. We hope to see lots of newcomers at this sprint, so we'll give friendly introductions. Note that during the Pycon conference we are giving PyPy talks which serve well as preparation. Goals and topics of the sprint ------------------------------ While attendees of the sprint are of course welcome to work on what they wish, we offer these ideas: - Work on an 'rctypes' module aiming at letting us use a ctypes implementation of an extension module from the compiled pypy-c. - Writing ctypes implementations of modules to be used by the above tool. - Experimenting with different garbage collection strategies. - Implementing Python 2.5 features in PyPy - Implementation of constraints solvers and integration of dataflow variables to PyPy. - Implement new features and improve the 'py' lib and py.test which are heavily used by PyPy (doctests/test selection/...). - Generally experiment with PyPy -- for example, play with transparent distribution of objects or coroutines and stackless features at application level. - Have fun! Location -------- The sprint will be held wherever the PyCon sprints end up being held, which is to say somewhere within the Dallas/Addison Marriott Quorum hotel. For more information see the PyCon 06 sprint pages: - http://us.pycon.org/TX2006/Sprinting - http://wiki.python.org/moin/PyCon2006/Sprints Exact times ----------- The PyPy sprint will from from Monday February 27th until Thursday March 2nd 2006. Hours will be from 10:00 until people have had enough. Registration, etc. ------------------ If you know before the conference that you definitely want to attend our sprint, please subscribe to the `PyPy sprint mailing list`_, introduce yourself and post a note that you want to come. Feel free to ask any questions or make suggestions there! There is a separate `PyCon 06 people`_ page tracking who is already planning to come. If you have commit rights on codespeak then you can modify yourself a checkout of http://codespeak.net/svn/pypy/extradoc/sprintinfo/pycon06/people.txt .. _`PyPy sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`PyCon 06 people`: http://codespeak.net/pypy/extradoc/sprintinfo/pycon06/people.txt From stephan.diehl at gmx.net Sat Feb 11 18:01:17 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Sat, 11 Feb 2006 18:01:17 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <20060210140933.GA30708@code0.codespeak.net> References: <20060210140933.GA30708@code0.codespeak.net> Message-ID: <200602111801.17413.stephan.diehl@gmx.net> Hi, since IRC is a bit impractical for error messages, I'm using this medium. I made some progress on the complex type: it's annotatable now. But I still can't compile it: The last lines I get, are: ---------------------------------------------------------------------------------------------------------------- [rtyper] specializing: 44900 / 45069 blocks (99%) [rtyper] specializing: 45000 / 45069 blocks (99%) [flowgraph] (pypy.rpython.lltypesystem.rclass:623)ll_issubclass [flowgraph] (pypy.rpython.lltypesystem.rclass:620)ll_type [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate_pypy.py", line 319, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/translator/driver.py", line 343, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/translator/tool/taskengine.py", line 108, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/translator/driver.py", line 140, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/translator/driver.py", line 186, in task_rtype [translation:ERROR] crash_on_first_typeerror=not opt.insist) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/rpython/rtyper.py", line 148, in specialize [translation:ERROR] self.exceptiondata.make_helpers(self) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/rpython/lltypesystem/exceptiondata.py", line 31, in make_helpers [translation:ERROR] self.fn_pyexcclass2exc = self.make_pyexcclass2exc(rtyper) [translation:ERROR] File "/home/stephan/projects/pypy-dist/pypy/rpython/lltypesystem/exceptiondata.py", line 79, in make_pyexcclass2exc [translation:ERROR] assert not clsdef.attrs, ( [translation:ERROR] AssertionError: should not have grown atributes [translation] batch mode, not calling interactive helpers --------------------------------------------------------------------------------------------------------------------------- I have no idea, where this is coming from. Maybe connected to this (or not, who knows) are a lot of warning like this: [rtyper:WARNING] prebuilt instance W_DictObject(r_dict({})) has no attribute 'imagval' [rtyper:WARNING] prebuilt instance W_DictObject(r_dict({})) has no attribute 'realval' I'm probably doing something wrong with the 'real', 'imag' properties that are defined in 'complextype.py'. I do believe (of course) that the code in question is right since it works when running pypy over cpython (but then, some other bugs didn't show when running over cpython as well) Any hints on what's going wrong are greatly appreciated. Cheers Stephan From stephan.diehl at gmx.net Sat Feb 11 18:04:24 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Sat, 11 Feb 2006 18:04:24 +0100 Subject: [pypy-dev] problems compiling with complex [2] Message-ID: <200602111804.24591.stephan.diehl@gmx.net> sorry, forgot to mention that pypy/objspace/std/model.py contains a flag WITHCOMPLEX which must be set to 'True' in order to use the new complex implementation. Stephan From tismer at stackless.com Sat Feb 11 20:04:27 2006 From: tismer at stackless.com (Christian Tismer) Date: Sat, 11 Feb 2006 20:04:27 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <200602111801.17413.stephan.diehl@gmx.net> References: <20060210140933.GA30708@code0.codespeak.net> <200602111801.17413.stephan.diehl@gmx.net> Message-ID: <43EE353B.9070609@stackless.com> Stephan Diehl wrote: ... > [translation:ERROR] AssertionError: 0xb7f9eadc> should not have grown atributes > [translation] batch mode, not calling interactive helpers OT: we should take the chance to correct this speling eror. ciao - chris :-) -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Sat Feb 11 21:00:11 2006 From: tismer at stackless.com (Christian Tismer) Date: Sat, 11 Feb 2006 21:00:11 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <200602111801.17413.stephan.diehl@gmx.net> References: <20060210140933.GA30708@code0.codespeak.net> <200602111801.17413.stephan.diehl@gmx.net> Message-ID: <43EE424B.1040001@stackless.com> Stephan Diehl wrote: > [translation:ERROR] AssertionError: 0xb7f9eadc> should not have grown atributes > [translation] batch mode, not calling interactive helpers > --------------------------------------------------------------------------------------------------------------------------- > > I have no idea, where this is coming from. > Maybe connected to this (or not, who knows) are a lot of warning like this: > > [rtyper:WARNING] prebuilt instance W_DictObject(r_dict({})) has no attribute > 'imagval' > [rtyper:WARNING] prebuilt instance W_DictObject(r_dict({})) has no attribute > 'realval' That's the typical effect when some object type was degenerated, and now it is assumed that every object has this attribute. You are probably doing some operation that does not make it clear to the objspace that you have a complex. You are also raising standard exceptions with argumens, which is not supported (thanks Samuele). I'd propose to use IRC if you need more hints. This is quicker than emails. There is not problem with pasting, since the relevant part is two lines. I guess you still want to solve it alone, right? ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Sat Feb 11 22:07:12 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 11 Feb 2006 22:07:12 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <43EE424B.1040001@stackless.com> References: <20060210140933.GA30708@code0.codespeak.net> <200602111801.17413.stephan.diehl@gmx.net> <43EE424B.1040001@stackless.com> Message-ID: <20060211210712.GA24966@code0.codespeak.net> Hi Christian, On Sat, Feb 11, 2006 at 09:00:11PM +0100, Christian Tismer wrote: > You are also raising standard exceptions with argumens, which > is not supported (thanks Samuele). Raising then is fine, in the sense that you can say raise ValueError("some message") but the message itself is lost in RPython, so you cannot read it back in complextype.py:159: except ValueError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) Also note that space.float_w() isn't supposed to raise ValueErrors, either. It should directly raise the correct OperationError, and you shouldn't have to worry about catching it. If it's not the case then something is wrong with float_w(). A bientot, Armin From tismer at stackless.com Sat Feb 11 22:44:43 2006 From: tismer at stackless.com (Christian Tismer) Date: Sat, 11 Feb 2006 22:44:43 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <20060211210712.GA24966@code0.codespeak.net> References: <20060210140933.GA30708@code0.codespeak.net> <200602111801.17413.stephan.diehl@gmx.net> <43EE424B.1040001@stackless.com> <20060211210712.GA24966@code0.codespeak.net> Message-ID: <43EE5ACB.9060103@stackless.com> Armin Rigo wrote: > Hi Christian, Wrong address. You are right, of course. >> You are also raising standard exceptions with argumens, which >> is not supported (thanks Samuele). > > Raising then is fine, in the sense that you can say > > raise ValueError("some message") > > but the message itself is lost in RPython, so you cannot read it back in > complextype.py:159: > > except ValueError, e: > raise OperationError(space.w_ValueError, space.wrap(e.msg)) > > Also note that space.float_w() isn't supposed to raise ValueErrors, > either. It should directly raise the correct OperationError, and you > shouldn't have to worry about catching it. If it's not the case then > something is wrong with float_w(). This is fine, but you shouldn't tell me, I'm just the trainer, here. -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Sat Feb 11 23:02:37 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 11 Feb 2006 23:02:37 +0100 Subject: [pypy-dev] problems compiling with complex In-Reply-To: <43EE5ACB.9060103@stackless.com> References: <20060210140933.GA30708@code0.codespeak.net> <200602111801.17413.stephan.diehl@gmx.net> <43EE424B.1040001@stackless.com> <20060211210712.GA24966@code0.codespeak.net> <43EE5ACB.9060103@stackless.com> Message-ID: <20060211220237.GA25937@code0.codespeak.net> Hi Christian, On Sat, Feb 11, 2006 at 10:44:43PM +0100, Christian Tismer wrote: > This is fine, but you shouldn't tell me, I'm just the trainer, here. This was a reply-to-all message sent to the list :-) You were left in the CC's by my client, as I usually do when replying to list messages. A bientot, Armin From Gerald.Klix at klix.ch Mon Feb 13 22:41:21 2006 From: Gerald.Klix at klix.ch (Gerald Klix) Date: Mon, 13 Feb 2006 22:41:21 +0100 Subject: [pypy-dev] Pointer To Member Problem with rctypes Message-ID: <43F0FD01.2040307@klix.ch> Hi all, I have a problem with GcStruct, Struct and Ptr I can not resolve with my own devices. After some trial with ctypes running on CPython I came up with the following lowlevel representation for structures: Ptr( GcStruct( "CtypesGcStructure_ ( "c_data" (Struct "C-Data_ *) ) ) ) Samuele and Armin helped to implement rtype_getattr and rtype_setattr that delegate all attribute access to the c_data field, by pointing out how to implement a helper method called generateCDataAccess: def generateCDataAccess( self, variable, lowLevelOperations ): """ Answer the C level data sub structure. """ inputargs = [ variable, inputconst( Void, "c_data" ) ] return lowLevelOperations.genop( "getfield", inputargs, # Using Ptr makes the C-compiler barf, # omitting Ptr leeds to specialisation errors Ptr( self.c_data_lowleveltype ) ) def rtype_setattr( self, highLevelOperation ): c_data = self.generateCDataAccess( highLevelOperation.inputarg( self, 0 ), highLevelOperation.llops ) inputargs = highLevelOperation.inputargs( *highLevelOperation.args_r[ :3 ] ) inputargs[ 0 ] = c_data print "inputargs:", inputargs print "r_result:", highLevelOperation.r_result highLevelOperation.genop( "setfield", inputargs ) Obviously generateCDataAccess needs to return a Ptr to the embedded C-structure. When I use Ptr as in the example above, I get the following traceback: c:\Dokumente und Einstellungen\root\Lokale Einstellungen\Temp\usession-353\testi ng_2\testing_2.c(894) : error C2440: '=': 'pypy_C_Data_tagpoint0' kann nicht in 'pypy_C_Data_tagpoint0 *' konvertiert werden Which means, that a 'pypy_C_Data_tagpoint0' can not be converted to a 'pypy_C_Data_tagpoint0 *'. A brief look at the generated C-source, shows that the compiler is right here: struct pypy_CtypesStructure_tagpoint0 *pypy_g__py_test_compile_struct(struct pypy_CtypesStructure_tagpoint0 *l_p_0, long l_x_1, long l_y_1) { /*void l_v111;*/ /*void l_v112;*/ struct pypy_C_Data_tagpoint0 *l_v98; struct pypy_C_Data_tagpoint0 *l_v99; struct pypy_CtypesStructure_tagpoint0 *l_v110; pypy_IncRf_CtypesStructure_tagpoint0(l_p_0); block0: l_v98 = l_p_0->cs_c_data; l_v98->cd_x = l_x_1; l_v99 = l_p_0->cs_c_data; l_v99->cd_y = l_y_1; l_v110 = l_p_0; goto block1; block1: return l_v110; } If I ommit the Ptr construction so that, generateCDataAccess is defined as follows: def generateCDataAccess( self, variable, lowLevelOperations ): """ Answer the C level data sub structure. """ inputargs = [ variable, inputconst( Void, "c_data" ) ] return lowLevelOperations.genop( "getfield", inputargs, self.c_data_lowleveltype ) I get the following error: [D:\svnwork\work\dist\pypy\rpython\lltypesystem\lltype.py:146] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def _nofield(self, name): raise AttributeError, 'struct %s has no field %r' % (self._name, E name) > AttributeError: struct C-Data_tagpoint has no field 'TO' This is no surprise at all. The getfield operation obviosly needs a pointer to work with. So it boils down to the question: How can I access the members of an embedded structure? Is there a possibility to get the address of substructure, just like C's ampersand (&) operator does? Any help is welcome. Best regards, Gerald From tismer at stackless.com Tue Feb 14 11:55:03 2006 From: tismer at stackless.com (Christian Tismer) Date: Tue, 14 Feb 2006 11:55:03 +0100 Subject: [pypy-dev] coroutine problem Message-ID: <43F1B707.806@stackless.com> Hi Armin, here a summary of my problem. This version compiles: def _bind(self, thunk): state = self.costate self.parent = state.current state.last.frame = yield_current_frame_to_caller() try: thunk.call() except CoroutineExit: # ignore a shutdown exception pass except Exception, e: # redirect all unhandled exceptions to the parent state.things_to_do = True state.temp_exc = e while self.parent is not None and self.parent.frame is None: # greenlet behavior is fine self.parent = self.parent.parent return self._update_state(state, self.parent) WHen I change it like this: def _bind(self, thunk): state = self.costate self.parent = state.current try: state.last.frame = yield_current_frame_to_caller() thunk.call() Then I get the exception mismatch. I still don't get why you think these are identical. It is true, the yield call does never create an exception by itself. But it is the place where I arrive from any other switch, and the switch is built in a way that it can create an exception. This pops up exactly at the place where we yield. So I'd like to show up inside an exception handler. Of course it is possible to move this handler further up. anything wrong with this reasoning? cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From pedronis at strakt.com Tue Feb 14 14:39:58 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Tue, 14 Feb 2006 14:39:58 +0100 Subject: [pypy-dev] Pointer To Member Problem with rctypes In-Reply-To: <43F0FD01.2040307@klix.ch> References: <43F0FD01.2040307@klix.ch> Message-ID: <43F1DDAE.6080106@strakt.com> Gerald Klix wrote: > Hi all, > I have a problem with GcStruct, Struct and Ptr I can not resolve with my > own devices. > > After some trial with ctypes running on CPython I came up with the > following lowlevel representation for structures: > > Ptr( GcStruct( "CtypesGcStructure_ > ( "c_data" > (Struct "C-Data_ > *) ) ) ) > to access c_data from a pointer to the surrounding GcStruct, you want the operation "getsubstruct", not "gefield" which is only for Primitive and pointer fields (not substructures), getsubstruct will return a pointer correctly. The low level operations for access are together: getfield / setfield getsubstruct (there's no setsubstruct) getarrayitem / setarrayitem getarraysubstruct (for array of structs) getarraysize and we have a cast_pointer also > Samuele and Armin helped to implement rtype_getattr and rtype_setattr > that delegate all attribute access to the c_data field, by pointing > out how to implement a helper method called generateCDataAccess: > > def generateCDataAccess( self, variable, lowLevelOperations ): > """ > Answer the C level data sub structure. > """ > inputargs = [ variable, inputconst( Void, "c_data" ) ] > return lowLevelOperations.genop( > "getfield", > inputargs, > # Using Ptr makes the C-compiler barf, > # omitting Ptr leeds to specialisation errors > Ptr( self.c_data_lowleveltype ) ) > > def rtype_setattr( self, highLevelOperation ): > c_data = self.generateCDataAccess( > highLevelOperation.inputarg( self, 0 ), > highLevelOperation.llops ) > inputargs = highLevelOperation.inputargs( > *highLevelOperation.args_r[ :3 ] ) > inputargs[ 0 ] = c_data > print "inputargs:", inputargs > print "r_result:", highLevelOperation.r_result > highLevelOperation.genop( "setfield", inputargs ) > > Obviously generateCDataAccess needs to return a Ptr to the embedded > C-structure. When I use Ptr as in the example above, I get the following > traceback: > > c:\Dokumente und Einstellungen\root\Lokale > Einstellungen\Temp\usession-353\testi > ng_2\testing_2.c(894) : error C2440: '=': 'pypy_C_Data_tagpoint0' > kann nicht in > 'pypy_C_Data_tagpoint0 *' konvertiert werden > > Which means, that a 'pypy_C_Data_tagpoint0' can not be converted to a > 'pypy_C_Data_tagpoint0 *'. A brief look at the generated C-source, > shows that the compiler is right here: > > struct pypy_CtypesStructure_tagpoint0 > *pypy_g__py_test_compile_struct(struct pypy_CtypesStructure_tagpoint0 > *l_p_0, long l_x_1, long l_y_1) { > /*void l_v111;*/ /*void l_v112;*/ > struct pypy_C_Data_tagpoint0 *l_v98; > struct pypy_C_Data_tagpoint0 *l_v99; > struct pypy_CtypesStructure_tagpoint0 *l_v110; > pypy_IncRf_CtypesStructure_tagpoint0(l_p_0); > > block0: > l_v98 = l_p_0->cs_c_data; > l_v98->cd_x = l_x_1; > l_v99 = l_p_0->cs_c_data; > l_v99->cd_y = l_y_1; > l_v110 = l_p_0; > goto block1; > > > block1: > return l_v110; > } > > If I ommit the Ptr construction so that, generateCDataAccess is defined > as follows: > > def generateCDataAccess( self, variable, lowLevelOperations ): > """ > Answer the C level data sub structure. > """ > inputargs = [ variable, inputconst( Void, "c_data" ) ] > return lowLevelOperations.genop( > "getfield", > inputargs, > self.c_data_lowleveltype ) > > I get the following error: > > [D:\svnwork\work\dist\pypy\rpython\lltypesystem\lltype.py:146] > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > def _nofield(self, name): > raise AttributeError, 'struct %s has no field %r' % > (self._name, > E name) > > AttributeError: struct C-Data_tagpoint has no field 'TO' > > This is no surprise at all. The getfield operation obviosly needs a > pointer to > work with. > > So it boils down to the question: > How can I access the members of an embedded structure? > Is there a possibility to get the address of substructure, just like > C's ampersand (&) operator does? > > > Any help is welcome. > > Best regards, > Gerald > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From arigo at tunes.org Tue Feb 14 17:27:01 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 14 Feb 2006 17:27:01 +0100 Subject: [pypy-dev] coroutine problem In-Reply-To: <43F1B707.806@stackless.com> References: <43F1B707.806@stackless.com> Message-ID: <20060214162701.GB12433@code0.codespeak.net> Hi Christian, On Tue, Feb 14, 2006 at 11:55:03AM +0100, Christian Tismer wrote: > state.last.frame = yield_current_frame_to_caller() > try: > thunk.call() > try: > state.last.frame = yield_current_frame_to_caller() > thunk.call() > > I still don't get why you think these are identical. > It is true, the yield call does never create an exception > by itself. > But it is the place where I arrive from any other > switch, and the switch is built in a way that it > can create an exception. This pops up exactly > at the place where we yield. This is wrong. The only two ways in which a stack switch occurs in RPython is by calling a switch() method explicitly, or by returning a frame object from a function that itself contained a call to yield_current_frame_to_caller(), like def _bind(). * if RPython code executes an expression 'f.switch()', it means that this RPython code is not currently propagating an RPython exception. If it were, it would not be executing any expression, it would be busy jumping back to immediate callers, or alternatively, busy checking if except statements match the current exception. Even if 'f.switch()' is in an except: block, the RPython exception has be caught when the switch() is executed, so there is no "current RPython exception" set any more. * if _bind() returns a frame, it will be switched to. This is if _bind() *returns* normally, so no exception. If, on the other hand, _bind() raises an exception without catching it, then it is a fatal error condition. This was all part of the original design of the interface in http://codespeak.net/svn/pypy/extradoc/sprintinfo/paris/stackless-discussion.txt """ a function that called ``yield_current_frame_to_caller()`` should not raise. It would have no implicit parent frame to propagate the exception to. That would be a crashingly bad idea. """ All this to say that there is no way you can pass an RPython exception "through" a yield or a switch. A bientot, Armin From tismer at stackless.com Tue Feb 14 19:54:10 2006 From: tismer at stackless.com (Christian Tismer) Date: Tue, 14 Feb 2006 19:54:10 +0100 Subject: [pypy-dev] coroutine problem In-Reply-To: <20060214162701.GB12433@code0.codespeak.net> References: <43F1B707.806@stackless.com> <20060214162701.GB12433@code0.codespeak.net> Message-ID: <43F22752.30400@stackless.com> Armin Rigo wrote: > This is wrong. The only two ways in which a stack switch occurs in > RPython is by calling a switch() method explicitly, or by returning > a frame object from a function that itself contained a call to > yield_current_frame_to_caller(), like def _bind(). I think you are right. The exception is happening in another place, my logic of kill() was imperfect. ciao & thanks - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From mwh at python.net Wed Feb 15 17:22:05 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Feb 2006 16:22:05 +0000 Subject: [pypy-dev] #pypy-sync tomorrow! Message-ID: <2md5hobniq.fsf@starship.python.net> Good afternoon! It is my pleasure to invite all active pypy developers to the weekly pypy-sync meeting which will take place tomorrow (the 16th of February) at 1pm CET (12 noon GMT) on the #pypy-sync channel on the freenode IRC network. The topics known at this point are: - three line activity reports as usual. - pycon sprint planning we should at the very least make sure our tutorial material is up to date before the pycon sprints. who will do this? - mozart/oz sprint some of the developers who are not going to this sprint are curious as to what the goals of this sprint are. a brief summary of what it's about would be appreciated. - extra core sprint we are probably going to want to have another core sprint before EuroPython, as both the PyCon and Tokyo sprints are likely to be rather beginner/dissemination oriented. where and when? i've heard that early april in leysin is a possibility. If there are any other topics you would like to discuss, please let me know by email. Cheers, mwh -- My hat is lined with tinfoil for protection in the unlikely event that the droid gets his PowerPoint presentation working. -- Alan W. Frame, alt.sysadmin.recovery From aurelien.campeas at logilab.fr Wed Feb 15 18:19:39 2006 From: aurelien.campeas at logilab.fr (=?ISO-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Wed, 15 Feb 2006 18:19:39 +0100 Subject: [pypy-dev] #pypy-sync tomorrow! In-Reply-To: <2md5hobniq.fsf@starship.python.net> References: <2md5hobniq.fsf@starship.python.net> Message-ID: <1140023980.12033.5.camel@musca.logilab.fr> Le mercredi 15 f?vrier 2006 ? 16:22 +0000, Michael Hudson a ?crit : [... snipped ...] > > If there are any other topics you would like to discuss, please let me > know by email. > I don't know the state (and feature set) of stackless integration into PyPy. I would like to know if there are plans to have (preemptible) microthreads, and if not, what can be done about it. Efficient threading is a key requirement for the parts of Oz we would like to see into PyPy. > Cheers, > mwh > From mwh at python.net Thu Feb 16 12:10:22 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 16 Feb 2006 11:10:22 +0000 Subject: [pypy-dev] Re: #pypy-sync tomorrow! References: <2md5hobniq.fsf@starship.python.net> Message-ID: <2m8xsbblup.fsf@starship.python.net> Michael Hudson writes: > > If there are any other topics you would like to discuss, please let me > know by email. Another topic: * moderation of pypy-sync meetings i will be unable to moderate the next pypy-sync meeting as i'll be on a plane (and for about 8 hours after it, too :/), and probably not the one after, as I'll still be at pycon. so we need someone to do those two, at least. Cheers, mwh -- Back in the old days, software would grow until it could send and receive e-mail, but now that even the virusses are doing that, the fashion has changed, and now software evolves until it has venomous fangs, the better to do serious damage when it sucks. -- AdB, asr From nicolas.chauvat at logilab.fr Thu Feb 16 12:55:59 2006 From: nicolas.chauvat at logilab.fr (Nicolas Chauvat) Date: Thu, 16 Feb 2006 12:55:59 +0100 Subject: [pypy-dev] web site improvements Message-ID: <20060216115559.GK22366@logilab.fr> Hi, I am willing to spend some hours improving the website. I understood that commiting changes to pypy/doc could be enough, but where is the top menu defined ? Changes I would like to make: - visible "events" info with sprints and conferences. - smaller font by default (css change would do) - visible link to SVN - shorter "getting started" getting to more info - simpler and more visible "here is what pypy is" section or page - probably others that would come to mind... -- Nicolas Chauvat logilab.fr - services en informatique avanc?e et gestion de connaissances From cfbolz at gmx.de Thu Feb 16 14:07:50 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 16 Feb 2006 14:07:50 +0100 Subject: [pypy-dev] web site improvements In-Reply-To: <20060216115559.GK22366@logilab.fr> References: <20060216115559.GK22366@logilab.fr> Message-ID: <43F47926.6010707@gmx.de> Hi Nicolas! Nicolas Chauvat wrote: > I am willing to spend some hours improving the website. cool! > I understood > that commiting changes to pypy/doc could be enough, Yes, indeed. If you run py.test in the doc directory, you can generate the whole website locally (and get failures if there are ReST bugs). See http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#changing-documentation-and-website for some more info. > but where is the > top menu defined ? in doc/confrest.py > Changes I would like to make: > > - visible "events" info with sprints and conferences. well, we have the pypy calendar, but I agree that it is not very visible. The news page usually announces current conferences but indeed having a separate page that completely lists all sprints/conferences maybe makes sense. > - smaller font by default (css change would do) I agree that PyPy has a particularly large font which I happen to like, but that is maybe not a very common taste. > - visible link to SVN yes. the svn repositories location is kind of hidden in getting-started. > - shorter "getting started" getting to more info I am not sure how shortening the getting-started document would improve matters. getting-started is the result of _a lot of_ effort and I think it is quite good (we also got positive feedback about it). It gives good entry points to playing with PyPy and the sections can be read independently (so "too long" is not an argument). > - simpler and more visible "here is what pypy is" section or page maybe adding it near the top of the architecture document would make sense? > - probably others that would come to mind... Thanks! Cheers, Carl Friedrich From mwh at python.net Thu Feb 16 14:30:22 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 16 Feb 2006 13:30:22 +0000 Subject: [pypy-dev] pypy-sync developer meeting 16th February 2006 Message-ID: <2m1wy3bfdd.fsf@starship.python.net> Time & location: 1pm (30 minutes), GMT+1 at #pypy-sync Attendees:: Carl Friedrich Bolz Anders Chrigstrom Aurelien Campeas Nik Haldimann Holger Krekel Samuele Pedroni Eric van Riet Paap Armin Rigo Michael Hudson (moderation & minutes) Regular Topics ==================== - activity reports (3 prepared lines of info). All Attendees submitted activity reports (see the `IRC Log`_ and 'LAST/NEXT/BLOCKERS' entries in particular) - resolve conflicts/blockers: No blockers Topics of the week =================== - pycon sprint planning Michael agreed to update our sprint tutorial materials. There was consensus that the "Paris style" of daily planning meetings and small-ish discussion groups presenting results was likely to be a good approach, although flexibility is required as we don't really know who's going to be there. Arre agreed to see if the Strakt projector could be brought along. - mozart/oz sprint Shortly before the meeting Nicolas checked in (revision 23397, pypy/extradoc/sprintinfo/louvain-la-neuve-2006) enough information to answer all questions :) - extra core sprint There was agreement that we need a core sprint before EuroPython. Armin suggested the middle or end of March in Leysin, but most other people preferred the beginning of April. Armin agreed to investigate a sprint location in Leysin for the dates 3-9th of April. So if you didn't attend the meeting, do want to attend the sprint and can't make these dates, now would be the time to shout :) - state of stackless The WP9 work requires an efficient threading implementation. Aurelien wanted to know if the stackless work had progressed to the point of making an efficient green threads implementation to be made. Armin said that anything was probably possible, but that precise details of what was needed were required. There was agreement that there should be a session on this at a sprint, or if that failed a planned session on IRC about the topic. The issue not especially time-critical, fortunately. - moderation of pypy-sync meetings There was consensus that too many people will be in the air to have a pypy-sync meeting next week. As most of those people will still be at pycon a week later for the next pypy-sync meeting, it's not clear that there will be one on the 2nd of March either. .. _`IRC log`: Complete IRC log ================= :: [12:55] ok, my clock says 12:02 [12:55] arigo (n=arigo at p54AB84C0.dip0.t-ipconnect.de) joined #pypy-sync. [12:55] so i'd like to start [12:55] activity reports, please [12:55] LAST: finished streams, scheduling problem [12:55] NEXT: exercise the solver with various problems, dbugging [12:55] BLOCK: nil [12:56] LAST: genc work [12:56] NEXT: more genc, pycon preparations [12:56] BLOCKERS: - [12:56] LAST: random explorations and fixes [12:56] NEXT: more explorations, also around rctypes and ootypesystem [12:56] BLOCKERS: none [12:56] LAST: stackless optimization and transformation, NEXT: more of that, BLOCKERS: - [12:56] LAST: codespeak refinements, press release, non-pypy [12:56] NEXT: pycon + sprints, customer-meeting [12:56] BLOCKERS: [12:56] PREV: JIT-work [12:56] NEXT: JIT-work [12:56] BLOCKERS: None [12:56] LAST: some GC work with Armin: finding the roots sort of works somewhat [12:56] NEXT: maybe more GC work [12:56] BLOCKERS: not enough time [12:57] arigo, pedronis ? [12:57] LAST: jit work, help with GC, some opt and polish, bug fixingwork [12:57] NEXT: jit,, pycon [12:57] BLOCKERS: - [12:57] LAST: Psyco stuff, GC framework progress with Carl [12:57] NEXT: more JIT, possibly more GC [12:57] BLOCKERS: - [12:57] ok [12:57] the first topic is pycon sprint planning [12:58] i'm not sure what we need to do other than update our tutorial [12:58] (also, we have some talks to write, i think...) [12:58] Action: hpk will take care for the py lib/py.test track of that sprint [12:58] i guess we are going to proceed similarly to paris [12:58] does anyone desperately want to work on the tutorial? [12:59] hpk: in which way do you mean? [12:59] mwh: having short "result presesntation and rough planning" daily sessions and otherwise work more in sub-groups [12:59] hpk: ah right [12:59] arre,pedronis: can we have the projector there? [12:59] i think we probably need to be flexbile (agile, even :) and see who turns up [13:00] yes [13:00] arigo: can you also prepare our architecture session talk a bit before the conference? [13:00] if noone else wants to work on the tutorial, i can do it [13:00] hpk: Provided Jacob does not want it I'll make sure to bring it allong. [13:00] mwh: would be great - i can review it at the conference but not before i guess [13:01] we've got a fair few topics to get though today, don't want to talk about this for two long [13:01] hpk: yes [13:01] so, decisions: arre to check about beamer, me to look at tutorial [13:01] any dissent? [13:01] no, fine [13:02] - mozart/oz sprint [13:02] i think for this we'd just like to hear dates, plans, who's going [13:02] nicolas just posted some bits of information about that [13:02] i take it this is an insiders only sprint [13:02] into svn [13:02] auc: ah [13:02] mwh: the sprint announcement does not sound like it [13:02] well, it can't hurt to do an announcement at least to pypy-dev? [13:02] cfbolz: ok [13:03] ah yes, i see the check in now [13:03] http://codespeak.net/svn/pypy/extradoc/sprintinfo/louvain-la-neuve-2006/ [13:03] ok, well i guess that check in answers all the questions [13:03] move on to next topic? [13:04] mwh: yes :-) [13:04] - extra core sprint [13:04] i don't think there's much doubt that we need another sprint before europython [13:05] as both pycon and tokyo will be dissemination focussed to some extent [13:05] and louvain-la-neuve is a bit off centre [13:05] mwh: agreed [13:05] there is a non-concrete idea about Leysin, but more mid-end-of-March than early April [13:05] arigo: i see [13:05] arigo: early april would not be possible? [13:05] i can't make the end of march [13:06] Action: hpk will not make it mid/end march either [13:06] the longer we wait, the less snow we have :-/ [13:06] arigo: I would also prefer early april :-) [13:06] arigo: hmm :) [13:06] i don't know if we want to try and decide dates now [13:06] just "yes, we need a sprint" [13:07] well, it seems that early april is more likely to fit at least with carl, michael and me [13:07] yes, we need a sprint [13:07] just time-wise [13:07] mwh: yes, we need a sprint [13:07] hpk: point taken [13:07] mid of march doesn't work for me either [13:07] auc: how about logilab? [13:07] don't know [13:07] it seems getting logilab and tismerysoft in the same place at some point would be a good thing [13:08] (see next topic...) [13:08] indeed [13:09] but april is better than march [13:09] ericvrp: how is beginning april for you? [13:09] I don't think I can make it to an intermediatte sprint [13:09] arigo: is leysin-in-april still a workable plan? [13:09] ericvrp: ok, i thought that because you don't come to pycon ... but then you'd like to go to tokyo, i see [13:09] (apart from the snow issue) [13:10] mwh: I propose that we fix potential dates (to be checked by me): [13:10] maybe I can come for a very short while, if that makes sense at all [13:10] the week of the 3rd-7th of April [13:10] arigo: yes, sounds like it [13:10] arigo: yes [13:10] and people can come around (preferably before) if they want to ski [13:10] if we don't fix dates now it will slip later Yet Another Time :-) [13:11] (predictably enough i'm prancing around in rock shoes the previous week...) [13:11] arigo: will you check out the old setting? [13:11] arigo: yes, agreed [13:11] 3-7? [13:11] (when's Easter?) [13:11] hpk: the same place, yes [13:11] easter sunday is 23rd april [13:11] i think [13:11] arigo: The week after. [13:11] 3-7th sounds a bit like no-breakday? [13:11] it's five days [13:12] arre: 16th? thanks [13:12] hpk: I talked about the week [13:12] ah ok [13:12] so we are talking about 3-9th with a breakday in between or so [13:12] er, yes 16th [13:12] yes, sounds good to me [13:12] 3-9 works for me [13:13] pedronis, arre: would work for you as well? [13:13] so, mention this on pypy-dev? [13:13] 3-9 is not a problem for me. [13:13] it works for me [13:13] mwh: ok (I need to check of course) [13:13] i'd like to move on to the next topic soon ish [13:13] arigo: of course [13:13] arigo: yip [13:13] fine with me, mwh: i think it's enough for now to mention it prominently in the minutes [13:14] hpk: ok [13:14] next topic: [13:14] - state of stackless [13:14] Action: ericvrp Christian needs to answer that one [13:14] this is auc's topic, but basically i think the question is particularly about whether there are concrete plans for microthreads [13:15] ericvrp: yeah [13:15] it's about 2am where he is though, i think [13:15] I think Christian's got coroutines, greenlets and tasklets so far [13:15] what are green/tasklets ? [13:15] wrt coroutines ? [13:15] and it should be very easy to add other abstractions [13:15] arigo: is that enough to build microthreads at app level? [13:15] auc: mostly just a different, slightly richer interface than bare coroutines [13:16] i guess not with preemption [13:16] mwh: sure [13:16] mwh: yes, it all depends on what is needed more precisely [13:16] indeed [13:16] anyway real threads are needed, if only because they are part of python, no ? [13:16] i guess pypy-sync isn't really the forum for this discussion in the long run [13:16] auc: well, yes. but you cannot have thousands of them [13:17] sure, that's why we want picrothreads [13:17] (shit) [13:17] it's not extremely urgent [13:17] picothreads? [13:17] auc: the answer is that it's mostly easy for any of us to build whatever you need, but it must be more well-defined [13:17] micro, nano, pico, femto, whatever ;-) [13:18] auc: good to know (about the urgency) [13:18] auc: well, would you want the exactly the same interface as threads, for example [13:18] arigo: it's simple, we want efficient threads [13:18] ideal would be to have a session on this at a sprint [13:18] bonus : threads that return values [13:18] auc: but for example, do you need threads that can block on system calls without blocking other threads [13:18] bonus : threads that expose the interface present in Oz [13:18] auc: I guess not [13:19] arigo: I guess yes [13:19] if not, would it make sense to schedule some irc session to get the relevant parties together? [13:19] auc: that's very hard to do without real OS threads [13:19] this interface is describe in some document I've committed already [13:19] mwh: yes [13:19] I can move it to doc/discussions [13:19] yes please [13:19] arigo: ok we'll see [13:20] so ideally this will be talked about at lln/leysin [13:20] otherwise irc sometime [13:20] now for two minutes: [13:20] who moderates the next pypy-sync? [13:20] it's not going to be me! [13:20] (auc: feel free to talk about in in #pypy at any time :-) [13:21] mwh: will there be enough people on next thursday? [13:21] aren't about half of the people present here going to be on planes next pypy-sync? [13:21] 23rd? half of us are in the plane [13:21] cfbolz: don't know [13:21] nikh: true [13:21] don't planes have wireless these days? [13:21] ok, so maybe there will not be a pypy-sync next week [13:21] that's fine [13:22] hpk: only if you pay $$$$ i think [13:22] arigo: yup [13:22] mwh: then let's plan the one afterwards [13:22] well, the one afterwards is on the day i'm travelling again [13:22] argh [13:22] it will be about 5am local time for pycon-ers, i think [13:23] hum [13:23] well, then let's not decide now in a hurry [13:23] ok [13:23] there are maybe 30 seconds for any final points :) [13:24] (we could take the opportunity to shift it to a different time) [13:24] consistency is good though [13:24] anyway, time up [13:24] Action: mwh closes the meeting -- 7. It is easier to write an incorrect program than understand a correct one. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From arigo at tunes.org Fri Feb 17 17:43:21 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 17 Feb 2006 17:43:21 +0100 Subject: [pypy-dev] Snake server downtime In-Reply-To: <20060131140751.GA28149@code0.codespeak.net> References: <20060131140751.GA28149@code0.codespeak.net> Message-ID: <20060217164321.GA14115@code0.codespeak.net> Hi, On Tue, Jan 31, 2006 at 03:07:51PM +0100, Armin Rigo wrote: > The "snake" server will be down during the next couple of days > (Wed-Thu), apparently for a change of the cooling system of the server > room. I'm told some people didn't figure out that it's back on-line. Indeed, it's been off-line for 4-5 days instead of 2, but still I expected by now that doing a 'ping' would have been easier than waiting for a mail from me :-) Will write an official notification next time, though. Sorry about the delay. A bientot, Armin From tismer at stackless.com Tue Feb 21 08:22:15 2006 From: tismer at stackless.com (Christian Tismer) Date: Mon, 20 Feb 2006 23:22:15 -0800 Subject: [pypy-dev] stackless patch Message-ID: <43FABFA7.7000208@stackless.com> Hi Eric, using your nice patch, I expected both a speed increase and a size decrease. The latter did not happen, because the exception is tested twice. #define StacklessUnwindAndRPyExceptionHandling(unwind_label, resume_label, exception_label) \ if (RPyExceptionOccurred()) { \ if (slp_frame_stack_bottom) \ goto unwind_label; \ resume_label: \ if (RPyExceptionOccurred()) \ FAIL(exception_label); \ } Just out of curiosity: does the code shrink if we do this? : #define StacklessUnwindAndRPyExceptionHandling(unwind_label, resume_label, exception_label) \ resume_label: \ if (RPyExceptionOccurred()) { \ if (slp_frame_stack_bottom) \ goto unwind_label; \ FAIL(exception_label); \ } Maybe this is slightly less machine code and not slower in the common case. Is it correct, btw? :-) ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From Ben.Young at risk.sungard.com Tue Feb 21 10:41:46 2006 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Tue, 21 Feb 2006 09:41:46 +0000 Subject: [pypy-dev] JIT Message-ID: Hi All, As you probably know by now, I enjoy following the progress of the PyPy project, and generally I can understand roughly what is going on, but I have to admit I am stumpt by the JIT. Is there a document that describes what is going on anywhere? What's the goal of the current bit of work? I can't actually work out if the current JIT for the toy language is actaully jitt'ed, or if you are currently just working on the data structures that will produce the JIT? I.e will there need to be a machine code backend for the JIT or does it somehow magically produce fast code without a compiler, using some other kind of dispatch loop? Cheers, Ben From mwh at python.net Tue Feb 21 13:26:52 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 21 Feb 2006 12:26:52 +0000 Subject: [pypy-dev] Re: JIT References: Message-ID: <2m8xs499tf.fsf@starship.python.net> Ben.Young at risk.sungard.com writes: > Hi All, > > As you probably know by now, I enjoy following the progress of the PyPy > project, and generally I can understand roughly what is going on, but I > have to admit I am stumpt by the JIT. I'm pretty lost too :) > Is there a document that describes what is going on anywhere? What's > the goal of the current bit of work? I can't actually work out if > the current JIT for the toy language is actaully jitt'ed, or if you > are currently just working on the data structures that will produce > the JIT? I.e will there need to be a machine code backend for the > JIT or does it somehow magically produce fast code without a > compiler, using some other kind of dispatch loop? This bit I can answer to some extent. Currently the JIT code produces L2 graphs, which are the low level graphs that can be translated to C. This is fine for the moment, but not in the long run, for two reasons: one is that RPython programs cannot manipulate L2 graphs (and eventually the JIT needs to be RPython so that it runs at a reasonable speed) and the other is that we don't want our JIT to have to compile it's generated code to C (or use llinterp...). So, we need something else. One option is indeed to generate machine code, but first we'll probably target another representation called L3 graphs (low low level graphs, if you like) which are simple enough to be manipulated and executed by RPython programs. The l3 interpreter/evaluator is partially written at this point, although some deep issues remain, particularly the interaction with garbage collection. Cheers, mwh -- Hiro dicks about with his computer, naturally. Being stranded on a life raft in the Pacific is a perfect venue for a hacker. -- Snow Crash, Neal Stephenson From eric at vanrietpaap.nl Wed Feb 22 22:43:19 2006 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Wed, 22 Feb 2006 22:43:19 +0100 Subject: [pypy-dev] Re: [pypy-svn] r23599 - pypy/dist/pypy/translator/c/src In-Reply-To: <20060222202249.351011007B@code0.codespeak.net> References: <20060222202249.351011007B@code0.codespeak.net> Message-ID: H, I have a local version that does the test for the 'errorvalue' that gets returned when stack unwinding is going on. I did not check this in yet because it did not give a speedup, I will keep it around and look into a little further. We probably are making life difficult for the compiler because of the use of the resume_label. What we IMO really should to do is tell the compiler not to bother with the label and let all the register setup be handled in the resume block. In other words we need to make it clearer (for the compiler) that non stack unwind/resume is the case to optimize. Now the question is of course, how to do that? cheers Eric On Feb 22, 2006, at 9:22 PM, tismer at codespeak.net wrote: > Author: tismer > Date: Wed Feb 22 21:22:46 2006 > New Revision: 23599 > > Modified: > pypy/dist/pypy/translator/c/src/ll_stackless.h > Log: > a tiny change to stackless switching. This is still optimizable by > using special return values (please talk with me when > considering). Anyway the effects of these changes are still rather > unpredictably. > > Modified: pypy/dist/pypy/translator/c/src/ll_stackless.h > ====================================================================== > ======== > --- pypy/dist/pypy/translator/c/src/ll_stackless.h (original) > +++ pypy/dist/pypy/translator/c/src/ll_stackless.h Wed Feb 22 > 21:22:46 2006 > @@ -26,14 +26,33 @@ > > #define RPyExceptionClear() rpython_exc_type = NULL > > +/* > #define StacklessUnwindAndRPyExceptionHandling(unwind_label, > resume_label, exception_label) \ > if (RPyExceptionOccurred()) { \ > if (slp_frame_stack_bottom) \ > goto unwind_label; \ > - resume_label: \ > + resume_label: \ > if (RPyExceptionOccurred()) \ > FAIL(exception_label); \ > } > + > + Following code was supposed to compiler to shorter machine > code, but on windows it doesn't. > + Probably some other code folding is prevented, and there is a > tiny increase of 20 kb. > + I'm leaving the change in here, anyway. Richards is getting a > bit slower, PySone > + is getting faster, all in all speed is slightly increased. > + We should further investigate and try to use Eric's suggestion > of checking certain > + return values to get even shorter code paths. > + In any case, these optimizations are still flaky, because we > are probably in a high > + noise level of caching effects and random decisions of the > compiler. > +*/ > +#define StacklessUnwindAndRPyExceptionHandling(unwind_label, > resume_label, exception_label) \ > + resume_label: \ > + if (RPyExceptionOccurred()) { \ > + if (slp_frame_stack_bottom) \ > + goto unwind_label; \ > + FAIL(exception_label); \ > + } > +*/ > #else > > #define RPyRaisePseudoException() > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn From eric at vanrietpaap.nl Wed Feb 22 23:07:27 2006 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Wed, 22 Feb 2006 23:07:27 +0100 Subject: [pypy-dev] Re: [pypy-svn] r23599 - pypy/dist/pypy/translator/c/src In-Reply-To: References: <20060222202249.351011007B@code0.codespeak.net> Message-ID: <7824E8FB-A0DF-4C4E-9460-5405D7E9DA4C@vanrietpaap.nl> HI, Sorry for replying to my own mail, but I will have a look into the gcc option -fprofile-arcs to aid branch prediction and maybe also experiment with __builtin_expect(...) to tell gcc that exception are not the common case. (sound logical doesn't?) . I take it similar options exist for other compilers, but that's another story... cheers Eric On Feb 22, 2006, at 10:43 PM, Eric van Riet Paap wrote: > H, > > I have a local version that does the test for the 'errorvalue' that > gets returned when stack unwinding is going on. I did not check > this in yet because it did not give a speedup, I will keep it > around and look into a little further. We probably are making life > difficult for the compiler because of the use of the resume_label. > What we IMO really should to do is tell the compiler not to bother > with the label and let all the register setup be handled in the > resume block. In other words we need to make it clearer (for the > compiler) that non stack unwind/resume is the case to optimize. Now > the question is of course, how to do that? > > cheers > Eric > > > On Feb 22, 2006, at 9:22 PM, tismer at codespeak.net wrote: > >> Author: tismer >> Date: Wed Feb 22 21:22:46 2006 >> New Revision: 23599 >> >> Modified: >> pypy/dist/pypy/translator/c/src/ll_stackless.h >> Log: >> a tiny change to stackless switching. This is still optimizable by >> using special return values (please talk with me when >> considering). Anyway the effects of these changes are still rather >> unpredictably. >> >> Modified: pypy/dist/pypy/translator/c/src/ll_stackless.h >> ===================================================================== >> ========= >> --- pypy/dist/pypy/translator/c/src/ll_stackless.h (original) >> +++ pypy/dist/pypy/translator/c/src/ll_stackless.h Wed Feb 22 >> 21:22:46 2006 >> @@ -26,14 +26,33 @@ >> >> #define RPyExceptionClear() rpython_exc_type = NULL >> >> +/* >> #define StacklessUnwindAndRPyExceptionHandling(unwind_label, >> resume_label, exception_label) \ >> if (RPyExceptionOccurred()) { \ >> if (slp_frame_stack_bottom) \ >> goto unwind_label; \ >> - resume_label: \ >> + resume_label: \ >> if (RPyExceptionOccurred()) \ >> FAIL(exception_label); \ >> } >> + >> + Following code was supposed to compiler to shorter machine >> code, but on windows it doesn't. >> + Probably some other code folding is prevented, and there is a >> tiny increase of 20 kb. >> + I'm leaving the change in here, anyway. Richards is getting a >> bit slower, PySone >> + is getting faster, all in all speed is slightly increased. >> + We should further investigate and try to use Eric's >> suggestion of checking certain >> + return values to get even shorter code paths. >> + In any case, these optimizations are still flaky, because we >> are probably in a high >> + noise level of caching effects and random decisions of the >> compiler. >> +*/ >> +#define StacklessUnwindAndRPyExceptionHandling(unwind_label, >> resume_label, exception_label) \ >> + resume_label: \ >> + if (RPyExceptionOccurred()) { \ >> + if (slp_frame_stack_bottom) \ >> + goto unwind_label; \ >> + FAIL(exception_label); \ >> + } >> +*/ >> #else >> >> #define RPyRaisePseudoException() >> _______________________________________________ >> pypy-svn mailing list >> pypy-svn at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-svn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From tismer at stackless.com Thu Feb 23 02:43:04 2006 From: tismer at stackless.com (Christian Tismer) Date: Wed, 22 Feb 2006 17:43:04 -0800 Subject: [pypy-dev] stackless optimization is somehow wrong Message-ID: <43FD1328.8060905@stackless.com> Hi Eric, I ran the whole stackless tests again. First I used my tiny optimisation, but the fifth test crashes after eating all memory. Then I switched back to your latest version, with the same result. Then I disabled the patch completely, and things worked, again. We must be missing something, no idea what. As an addition, the fifteenth test always crashes with too many heap sections (on windows). ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From rxe at ukshells.co.uk Thu Feb 23 07:12:17 2006 From: rxe at ukshells.co.uk (Richard Emslie) Date: Thu, 23 Feb 2006 06:12:17 +0000 (GMT) Subject: [pypy-dev] Re: [pypy-svn] r23602 - in pypy/dist/pypy: rpython/memory rpython/memory/test translator/c In-Reply-To: <20060222213925.0286F1007C@code0.codespeak.net> References: <20060222213925.0286F1007C@code0.codespeak.net> Message-ID: On Wed, 22 Feb 2006 mwh at codespeak.net wrote: > Author: mwh > Date: Wed Feb 22 22:39:23 2006 > New Revision: 23602 > > Modified: > pypy/dist/pypy/rpython/memory/gctransform.py > pypy/dist/pypy/rpython/memory/test/test_gctransform.py > pypy/dist/pypy/translator/c/database.py > pypy/dist/pypy/translator/c/extfunc.py > pypy/dist/pypy/translator/c/node.py > Log: > rework the gc transformer to use the mixlevelannotator and do all its rtyping > in one big go at the end. the hard part of this is attacking genc such that it > gets all its annotating done before the end of db.complete() and avoids looking > too hard at any of the gc helpers until after they get rtyped. > > this almost certainly breaks llvm :/ > oops! :-) Well I am now attending the sprint at pycon - would like to take a look at this and hopefully have a chance to get an understanding of gctransform. It is quite nice a dependency when something changes here and breaks llvm, prompts me to have a look into what has going on. ;-) Cheers, Richard From mwh at python.net Thu Feb 23 07:13:44 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 23 Feb 2006 06:13:44 +0000 Subject: [pypy-dev] Re: [pypy-svn] r23609 - pypy/dist/pypy/translator/c/src In-Reply-To: <20060223014834.3EFC310088@code0.codespeak.net> (tismer@codespeak.net's message of "Thu, 23 Feb 2006 02:48:34 +0100 (CET)") References: <20060223014834.3EFC310088@code0.codespeak.net> Message-ID: <2mpsle7gbr.fsf@starship.python.net> tismer at codespeak.net writes: > Author: tismer > Date: Thu Feb 23 02:48:31 2006 > New Revision: 23609 > > Modified: > pypy/dist/pypy/translator/c/src/ll_stackless.h > Log: > had to disable Eric's stackless optimization. The fifth test on > test_stackless.py crashes after utilizing all memory. This error > should have been caught by running tests. How comes? Just py.test pypy/translator/c/test_stackless.py ? I ran that at least 10 times yesterday with no failures... Cheers, mwh -- TRSDOS: Friendly old lizard. Or, at least, content to sit there eating flies. -- Jim's pedigree of operating systems, asr From mwh at python.net Thu Feb 23 09:46:50 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 23 Feb 2006 08:46:50 +0000 Subject: [pypy-dev] Re: [pypy-svn] r23602 - in pypy/dist/pypy: rpython/memory rpython/memory/test translator/c References: <20060222213925.0286F1007C@code0.codespeak.net> Message-ID: <2mlkw2798l.fsf@starship.python.net> Richard Emslie writes: > On Wed, 22 Feb 2006 mwh at codespeak.net wrote: > >> Author: mwh >> Date: Wed Feb 22 22:39:23 2006 >> New Revision: 23602 >> >> Modified: >> pypy/dist/pypy/rpython/memory/gctransform.py >> pypy/dist/pypy/rpython/memory/test/test_gctransform.py >> pypy/dist/pypy/translator/c/database.py >> pypy/dist/pypy/translator/c/extfunc.py >> pypy/dist/pypy/translator/c/node.py >> Log: >> rework the gc transformer to use the mixlevelannotator and do all its rtyping >> in one big go at the end. the hard part of this is attacking genc such that it >> gets all its annotating done before the end of db.complete() and avoids looking >> too hard at any of the gc helpers until after they get rtyped. >> >> this almost certainly breaks llvm :/ >> > oops! :-) Well I am now attending the sprint at pycon - would like to > take a look at this and hopefully have a chance to get an > understanding of gctransform. It is quite nice a dependency when > something changes here and breaks llvm, prompts me to have a look into > what has going on. ;-) Well, it's not the gctransformer stuff that broke llvm, it's my changes to translator/c/extfunc.py... you're entirely welcome to look into gctransformer, of course! See you later? (I land 1620 local time) Cheers, mwh (typing over a annoyingly expensive and high latency ariport wifi connection...) -- No. In fact, my eyeballs fell out just from reading this question, so it's a good thing I can touch-type. -- John Baez, sci.physics.research From eric at vanrietpaap.nl Sat Feb 25 21:59:28 2006 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Sat, 25 Feb 2006 21:59:28 +0100 Subject: [pypy-dev] profile based compiler optimization for PyPy Message-ID: Hi, I've been experimenting with with some gcc options and features. What I found out so far is that with the options -fprofile-generate and - fprofile-use I was able to increase the Richards and pystone benchmark results (on top of pypy-c) with a little over 15%. Stackless is currently still 10-15% slower compared to plain vanilla PyPy, but I have some idea that I would like to look into in the next couple of days to make that better. cheers Eric From Gerald.Klix at klix.ch Mon Feb 27 22:43:53 2006 From: Gerald.Klix at klix.ch (Gerald Klix) Date: Mon, 27 Feb 2006 22:43:53 +0100 Subject: [pypy-dev] Request for Review Message-ID: <44037299.8080505@klix.ch> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, can someone with more knowledge than me - I am thinking of Samuele and Armin here - look at pypy/dist/pypy/doc/ctypes-integration.txt and tell me and the other ctypes workers, wether the pointer representation is translateable? Currently I am running into problems when I want to translate a function that returns a pointer to a strcuture. Michael gave a hint, telling me that returning a pointer won't work with the test wrapper., but if I change the function to return something differnt (False) the problem persists. I am talking about the tests in pypy/dist/pypy/rpython/rctypes/test/test_rctypes.py around line 423 (def test_compile_pointer_to_struct). Any help is welcome. Best regards, Gerald - -- GPG-Key: http://keyserver.veridis.com:11371/search?q=0xA140D634 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFEA3KYEDg9cqFA1jQRAq46AJ4ormuSdskdAsIyIJM4lh9brcWv/wCfWdGj Qzet6bd2Zp36+vfK2+DZJ34= =zsrh -----END PGP SIGNATURE----- From aurelien.campeas at logilab.fr Tue Feb 28 09:57:37 2006 From: aurelien.campeas at logilab.fr (=?ISO-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Tue, 28 Feb 2006 09:57:37 +0100 Subject: [pypy-dev] Re: [pypy-svn] r23678 - pypy/extradoc/talk/pycon2006 In-Reply-To: <20060226152716.C2977100A7@code0.codespeak.net> References: <20060226152716.C2977100A7@code0.codespeak.net> Message-ID: <1141117057.4899.7.camel@octans> On Sun, 2006-02-26 at 16:27 +0100, bea at codespeak.net wrote: [...] > +- Sprints: Lovain La Neuve, Tokyo, EuroPython, Ireland that's Louvain-La-Neuve Cheers, Aur?lien.