From paul at prescod.net Mon Jun 5 01:31:33 2006 From: paul at prescod.net (Paul Prescod) Date: Sun, 4 Jun 2006 19:31:33 -0400 Subject: [pypy-dev] Vancouver Python Workshop Message-ID: <1cb725390606041631r263b38f5r904ad3b0b6a59b0d@mail.gmail.com> We would be very appreciative if one or more of you PyPy experts would come to the Vancouver Python Workshop and inform of us of your progress on this exciting project. For invited talks like this, we offer a substantially reduced discount on our already minimal registration fees. In addition, we already have some really good talks organized on non-PyPy topics, including Python 3000 and IronPython, so the conference should be fun for whoever decides to come. http://www.vanpyz.org/conference Paul Prescod Vancouver Python Workshop -------------- next part -------------- An HTML attachment was scrubbed... URL: From bea at changemaker.nu Tue Jun 6 13:36:51 2006 From: bea at changemaker.nu (Beatrice During) Date: Tue, 06 Jun 2006 13:36:51 +0200 Subject: [pypy-dev] Vancouver Python Workshop In-Reply-To: <1cb725390606041631r263b38f5r904ad3b0b6a59b0d@mail.gmail.com> References: <1cb725390606041631r263b38f5r904ad3b0b6a59b0d@mail.gmail.com> Message-ID: <448568D3.8060806@changemaker.nu> Hi there Paul Prescod skrev: > We would be very appreciative if one or more of you PyPy experts would > come to the Vancouver Python Workshop and inform of us of your progress > on this exciting project. Thanks for inviting us! > For invited talks like this, we offer a substantially reduced discount > on our already minimal registration fees. In addition, we already have > some really good talks organized on non-PyPy topics, including Python > 3000 and IronPython, so the conference should be fun for whoever decides > to come. > > http://www.vanpyz.org/conference > > Paul Prescod > Vancouver Python Workshop > We are checking whether some one of us can participate - we will get back to you as soon as possible. Cheers Beatrice D?ring > ------------------------------------------------------------------------ > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From mwh at python.net Wed Jun 7 12:49:57 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 07 Jun 2006 11:49:57 +0100 Subject: [pypy-dev] Duesseldorf Sprint Report Days 1-4 Message-ID: <2mverdqlyi.fsf@starship.python.net> Hi pypy-dev! So we're sitting in the geography^Wcomputer science department of Heinrich-Heine-Universitaet Duesseldorf, the alleged employers of Armin and Michael, and (also alleged) educator of Carl. We've been doing the usual sprinty things of drinking beer, getting up late, going to bed even later (and so still not really getting much sleep) and hacking. An early task was finishing off the mostly complete weakref support. After drawing an extremely confusing diagram: http://codespeak.net/~mwh/blackboard.jpg Carl and Arre realized that there were a few strange cases that our model cannot support (in the case of a object and a weakref-with-callback to the same object both being in the same trash cycle, it is unclear whether the callback will be called). We plan to deal with this by calling anyone who claims to care insane. They then looked at getting CPython's test_weakref to run (something that's been on the "that would be nice" list for a looong time). This uncovered an amusing problem, a result of three facts: 1. the conservative Boehm collector cannot distinguish between a pointer and an integer that happens to have the same bit pattern as the pointer (this is what is meant by "conservative"). 2. the hash of a weakref object is the hash of the referenced object. 3. the default hash of a Python instance is it's id, which is usually the memory address cast to an int. This made weakref.WeakKeyDictionary rather poorly named. This was fixed by changing the id when using Boehm; the pointer is cast to an integer and then bitwise complemented. Accidentally, this means that that this: bool(id(object()) & 1) is now a check for pypy-c linked with the Boehm GC (use this fact with care). After solving the weak-keys-are-strong problem and adding a few calls to gc.collect() to CPython's test_weakref (to give Boehm a hurry up), test_weakref now passes. This leads on to the work that Anders and Holger did during the first day of the sprint: improving PyPy's "compliance score": http://codespeak.net/~hpk/pypy-testresult/ After a couple of months of relative neglect, this measure of how accurately we implement CPython's behaviour had slipped to around 80%. After fixing a few problems with the new interp-level implementation of the complex type and the above weakref work, we are back well over 95%. Michael helped Samuele remember exactly what he was thinking of in a branch that previously had only been the subject of hacking in the small hours of the night before his vacation. The goal of the nocturnal head bashing was to introduce the concept of an "explicit resume point" in an RPython program: a named, arbitrary location in the code and a way of transferring control to that point together with values for all live local variables. For example: def f(x, y, z): r = x + y rstack.resume_point("rp_f", r, z) return r + z defines a resume point called "rp_f" and records that the values of r and z must be supplied when the state is jumped to, like so: state = rstack.resume_state_create(None, "rp_f", 3, 4) result = rstack.resume_state_invoke(state) Something like this is a necessary piece in the puzzle that is implementing tasklet pickling, or more properly tasklet unpickling. This was a nice illustration of the "build one to throw away" concept: we were able to see that what Samuele had implemented as one operation really needed to be implemented as two (resume_state_create and resume_state_invoke in the above examples), and the rest of the coding only required moderate amounts of head bashing. By this morning, they had written a test that manually copied a (interp-level) coroutine, which validates the approach to at least some extent. You can also use explicit resume points to write some extremely confusing programs (as we discovered by mistake when debugging). The other major pieces of the tasklet pickling puzzle are being able to pickle and unpickle the core interpreter types such as frames, generators, tracebacks, ..., placing enough explicit resume points in the source of the Standard Interpreter to be able to resume a pickled tasklet and integration of the various pieces. Christian and Eric (working remotely; he arrives in Duesseldorf today) worked on the pickling and unpickling of interpreter types and have now implemented this for enough types that we now have no real option but to face the scary task of making something useful. For the last couple of days, the attention of Holger, Armin (still recovering from the 'flu, that's why we haven't mentioned him yet), Arre and a bit of Carl have been focussed on the elusive "ext-compiler": a tool designed to take an RPython module in a form suitable for making an "extension" for PyPy and translating it into an efficient extension module for CPython. This would clearly already be useful for purely algorithmic code, but the new-ish rctypes package of PyPy allows it to be used for modules that wrap an external library. While most of the basic technology is present for this, the tool is still far too rough to be unleashed even on unsuspecting Summer of Code students, never mind mere humans. Holger and Arre fleshed out ideas for how to build a more usable interface and wrote some documentation explaining this interface (who said open source projects are rubbish at documentation?). Armin began implementing support for typedefs, i.e. exposing custom types in the module interface, but ran into a small army of confusing levels of abstraction, which shift around as soon as you aren't staring at them. We're sure they're no match for Armin's planet-sized [#]_ brain though :) .. [#] http://mail.python.org/pipermail/python-dev/2003-January/032600.html While some of the above work involved much discussion and occasional voluminous cursing, two or three people have been sitting in the corner muttering about strange things like "ootypes" and "dot-net" and "javaskript", as well as producing a stream of high quality checkins (most of them have the log message "More ootypesystem tests", though). PyPy semi-regular Nik and newcomers and Summer of Code participants Antonio and Maciej worked on the ootypesystem backends, fixing many bugs and implementing new features, to the point that the richards and rpystone benchmarks can be run using the llinterpreter (now increasingly badly named; rinterpreter would prooooobably be better, but would have other unhelpful connotations). They couldn't run these benchmarks without a small modification: removing the print at the end that tells you how fast they ran (running things on the llinterpreter is so horrendously slow that this doesn't defeat the point of the excercise as you might at first think). The problem with printing is that it involves calling an external function, something that was not at all supported in the ootypesystem's world. So they decided to fix that :) And they did. Maciej also worked on the newer, ootype-using version of Javascript backend, and can now write python programs which bounce bub-n-bros characters around a browser window. This was a good way of getting Armin's attention :) Pozdrawiam, mwh & Carl Friedrich -- "Well, the old ones go Mmmmmbbbbzzzzttteeeeeep as they start up and the new ones go whupwhupwhupwhooopwhooooopwhooooooommmmmmmmmm." -- Graham Reed explains subway engines on asr From holger at merlinux.de Wed Jun 7 13:33:19 2006 From: holger at merlinux.de (holger krekel) Date: Wed, 7 Jun 2006 13:33:19 +0200 Subject: [pypy-dev] PyPy post-EP sprint (6-9th july) Message-ID: <20060607113319.GP25313@solar.trillke> Geneva/CERN Post-EuroPython PyPy sprint 6-9th July ==================================================== The next PyPy sprint will be held at CERN/Geneva right after the EuroPython conference. We specifically invite newcomers and people interested to get to know and get into PyPy development. We are going to work on a variety of topics and will give introductions to various aspects of PyPy (on the 6th morning). Some foreseen focus will be on the following topics: * The extension module compiler. Using it for implementing extension modules both for PyPy and CPython from the same source code. (for example if you have a Need for Speed :) * work and experiment with our to-be-relased 0.9 stackless features * work on high level backends: .NET, Javascript etc. * optimization of core Python data types, making full use of PyPy's flexible architecture and python-implemented (and then translated) type system. * You may even dare to dive into ongoing work on the JIT compiler * experimenting with novel security systems for Python, enabled by PyPy * and so on :) Registration ---------------------- If you'd like to come, please subscribe to the `pypy-sprint mailing list`_ and drop a note about your interests and post any questions. More organisational information will be send to that list. We'll keep a list of `people`_ which we'll update (which you can do so yourself if you have codespeak commit rights). Special note to Students! ---------------------------- As you might know, there are three students in Google's Summer of Code program who are working on projects related to PyPy. In addition we are planning our own "Summer of PyPy": we can cover the expenses of attending some of our sprints and provide mentorship. In this vein, if you have an interesting idea for a project relating to PyPy you should send us ("pypy-tb at codespeak.net") a proposal. We will consider it get back to you on whether we can fund your travel and accomodation. We expect to able to fund between 4 and 6 students in this way. As we still need to figure out budget/money issues, for now we can only promise for now that we can fund this Post-EP2006 sprint for such students. But don't hesitate to come to our irc channel, ask around and produce proposals! .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`people`: people.html From tismer at stackless.com Thu Jun 8 11:34:07 2006 From: tismer at stackless.com (Christian Tismer) Date: Thu, 08 Jun 2006 11:34:07 +0200 Subject: [pypy-dev] The sound of a sprint :-) Message-ID: <4487EF0F.8040509@stackless.com> http://tismerysoft.de/~tismer/movies/sprint-ddorf/CIMG0648.AVI recorded in the backyard of building 25.12 of HHU Duesseldorf. You go through the cafeteria and pass two hallways, then there is an open air space with all this sound Python food. From bea at changemaker.nu Thu Jun 8 11:52:26 2006 From: bea at changemaker.nu (Beatrice During) Date: Thu, 08 Jun 2006 11:52:26 +0200 Subject: [pypy-dev] The sound of a sprint :-) In-Reply-To: <4487EF0F.8040509@stackless.com> References: <4487EF0F.8040509@stackless.com> Message-ID: <4487F35A.1090005@changemaker.nu> Hi there Christian Tismer skrev: > http://tismerysoft.de/~tismer/movies/sprint-ddorf/CIMG0648.AVI > > recorded in the backyard of building 25.12 of HHU Duesseldorf. > You go through the cafeteria and pass two hallways, then > there is an open air space with all this sound Python food. So if you kiss the frogs they turn into Armin and Christian and.... ;-) Good to hear that the sprint is productive - although I must say I did not understand everything that was said ;-) Cheers Bea From tismer at stackless.com Thu Jun 8 15:09:15 2006 From: tismer at stackless.com (Christian Tismer) Date: Thu, 08 Jun 2006 15:09:15 +0200 Subject: [pypy-dev] The sound of a sprint :-) In-Reply-To: <4487F35A.1090005@changemaker.nu> References: <4487EF0F.8040509@stackless.com> <4487F35A.1090005@changemaker.nu> Message-ID: <4488217B.8050609@stackless.com> Beatrice During wrote: > So if you kiss the frogs they turn into Armin and Christian and.... Aah. This was the missing spot. Of course, this is the picture. A bewitched sprint in the magic wood. Thinking to write a small poem :-> But I think *you* need to do the kissing. If I did it you would probably get a shaving machine or a fridge... ciao - quaaa quaaa bregegegex -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Fri Jun 9 10:24:50 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 9 Jun 2006 10:24:50 +0200 Subject: [pypy-dev] Core sprint before EuroPython Message-ID: <20060609082450.GA10125@code0.codespeak.net> Hi all, ------------------------- Leysin Summer 2006 sprint ------------------------- As mentionned previously, there will be another sprint in Switzerland just before EuroPython. This one is meant for more "core" people in the broad sense, i.e. anyone that has already some knowledge about PyPy. For newcomers we recommend the post-EuroPython sprint at the conference venue :-) The main sprint focus will be on the JIT, but we can also work on other areas, as usual. This pre-EP sprint will take place from the 28th of June (included), to the 2nd of July (probably excluded, to have a break and travel to Geneva). The location is Leysin, about 1h45 away from Geneva by train. The sprint place is the same as during the previous Leysin sprints: Chalet Ermina, http://www.ermina.ch/ . I am arranging a group booking as usual. List of participants (add yourself there): http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-summer-2006/people.txt A bientot, Armin From tismer at stackless.com Sat Jun 10 16:53:18 2006 From: tismer at stackless.com (Christian Tismer) Date: Sat, 10 Jun 2006 16:53:18 +0200 Subject: [pypy-dev] [pypy-svn] r28615 - in pypy/dist/pypy/module/stackless: . test In-Reply-To: <20060610130802.DAA3510053@code0.codespeak.net> References: <20060610130802.DAA3510053@code0.codespeak.net> Message-ID: <448ADCDE.2000702@stackless.com> arigo at codespeak.net wrote: > Author: arigo > Date: Sat Jun 10 15:07:59 2006 > New Revision: 28615 > > Modified: > pypy/dist/pypy/module/stackless/interp_coroutine.py > pypy/dist/pypy/module/stackless/test/test_interp_coroutine.py > Log: > Long test and one-liner fix for coroutines and app-coroutines, Ah, well. This is a typical one. It is always hard to tell "who am I" in the coroutine stuff. By default, it is someone else :-) -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From stephan.diehl at gmx.net Mon Jun 12 19:25:39 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Mon, 12 Jun 2006 19:25:39 +0200 Subject: [pypy-dev] christian is ill Message-ID: <448DA393.1030207@gmx.net> Hi all, I've just talked with Christian. Unfortunatelly, he's ill and at a hospital. He won't be able to work for a couple of days. I guess that it's a good bet that he won't be available before friday. Cheers Stephan From mwh at python.net Tue Jun 13 00:11:46 2006 From: mwh at python.net (Michael Hudson) Date: Mon, 12 Jun 2006 23:11:46 +0100 Subject: [pypy-dev] pypy-c vs cpython tests Message-ID: <2md5deowgt.fsf@starship.python.net> I've spent an aggravating little while running CPython's regrtest with a recent pypy-c (default options, so boehm and no fancy stuff). After a bit of trial and error, this was the result: 69 tests failed: GC Warning: Finalization cycle involving d66c120 GC Warning: Finalization cycle involving a8e06c0 GC Warning: Finalization cycle involving d286d08 GC Warning: Finalization cycle involving d180780 GC Warning: Finalization cycle involving d12a108 test___all__ test_anydbm test_calendar test_cfgparser test_class test_codecencodings_cn test_codecencodings_hk test_codecencodings_jp test_codecencodings_kr test_codecencodings_tw test_coercion test_commands test_compare test_compiler test_cookielib test_cpickle test_decimal test_descr test_descrtut test_dis test_distutils test_doctest2 test_dumbdbm test_extcall test_file test_filecmp test_fileinput test_frozen test_gc test_getargs test_gettext test_glob test_inspect test_iter test_mailbox test_marshal test_mhlib test_mimetools test_minidom test_multibytecodec test_new test_os test_peepholer test_pep292 test_pickle test_pickletools test_posixpath test_profile test_pyclbr test_random test_repr test_scope test_sort test_strftime test_stringprep test_strptime test_structseq test_tarfile test_tempfile test_time test_traceback test_transformer test_ucn test_unicodedata test_weakref test_whichdb test_xmlrpc test_xpickle test_zipfile 97 tests skipped: test__locale test_aepack test_al test_applesingle test_asynchat test_audioop test_bsddb test_bsddb185 test_bsddb3 test_bz2 test_capi test_cd test_cgi test_cl test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_crypt test_csv test_curses test_dbm test_dl test_doctest test_email test_email_codecs test_fcntl test_fork1 test_gdbm test_getargs2 test_gl test_grp test_gzip test_hotshot test_httplib test_imageop test_imaplib test_imgfile test_imp test_ioctl test_largefile test_linuxaudiodev test_locale test_logging test_macfs test_macostools test_mimetypes test_mmap test_nis test_normalization test_openpty test_ossaudiodev test_pep277 test_plistlib test_poll test_popen test_popen2 test_pty test_pwd test_pyexpat test_queue test_regex test_resource test_rgbimg test_robotparser test_sax test_scriptpackages test_select test_signal test_site test_socket test_socket_ssl test_socketserver test_strop test_subprocess test_sunaudiodev test_sundry test_symtable test_tcl test_thread test_threaded_import test_threadedtempfile test_threading test_threading_local test_threadsignals test_timeout test_timing test_unicode_file test_urllib test_urllib2 test_urllib2net test_urllibnet test_winreg test_winsound test_zipimport test_zlib Segmentation fault I had to run with '-x test_datetime test_format test_shutil test_shelve test_str test_string test_unicode test_userstring' in the end. test_datetime, test_shutil and test_format kill the build. the string tests fail in ways that suggest that the boehm gc really didn't expect the usage patterns they use. I don't remember why I skipped test_shelve now :) I count around 250 test_* files in CPython's test suite, so we actually pass around 40% of them, and a lot of the skips and plenty of the fails are missing (parts of) extension modules, so we're not really in *that* bad a shape. But fixing the crashers would be nice. We need to get some automated version of this set up. Cheers, mwh -- ARTHUR: Why should he want to know where his towel is? FORD: Everybody should know where his towel is. ARTHUR: I think your head's come undone. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From mwh at python.net Tue Jun 13 09:52:24 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 13 Jun 2006 08:52:24 +0100 Subject: [pypy-dev] pypy-c vs cpython tests References: <2md5deowgt.fsf@starship.python.net> Message-ID: <2m3be9pk5j.fsf@starship.python.net> Michael Hudson writes: > I've spent an aggravating little while running CPython's regrtest with > a recent pypy-c (default options, so boehm and no fancy stuff). > > After a bit of trial and error, this was the result: > > 69 tests failed: > 97 tests skipped: In some good news, the stacklessgc build passes exactly one fewer test (test_builtin, known issue) and doesn't even do this: > Segmentation fault and the end :) Though it does exit with a rather strange ImportError: No module named _socket Haven't dug yet. On this evidence, I think I'm inclined to say that framework GC builds are at least as solid as the Boehm builds. Cheers, mwh -- C is not clean -- the language has _many_ gotchas and traps, and although its semantics are _simple_ in some sense, it is not any cleaner than the assembly-language design it is based on. -- Erik Naggum, comp.lang.lisp From hpk at trillke.net Thu Jun 15 07:56:49 2006 From: hpk at trillke.net (holger krekel) Date: Thu, 15 Jun 2006 07:56:49 +0200 Subject: [pypy-dev] 0.9 release meeting 11AM (GMT+2) Message-ID: <20060615055649.GV25313@solar.trillke> Hi folks, today at 11AM (GMT+2) we'll meet on #pypy-release for a meeting regarding the upcoming 0.9 release Topics will be: * extension compiler * stackless features * (high level backends / snapshot of work) * logic/constraint works * feature freeze (branching off?) * documentation about all this and whatever we decide to be relevant for it. best, holger From stephan.diehl at gmx.net Thu Jun 15 16:05:34 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Thu, 15 Jun 2006 16:05:34 +0200 Subject: [pypy-dev] stackless (the module) exception handling Message-ID: <4491692E.60306@gmx.net> this is all very confusing, but I'll do my best. The fun stuff is happening in interp_coroutine.py which looks at the moment like this: -------------------------------------------------------------- def _execute(self, incoming_frame): syncstate.switched(incoming_frame) state = self.costate try: try: self.thunk.call() finally: self.finished() self.thunk = None resume_point("coroutine__bind", self, state) except CoroutineExit: # ignore a shutdown exception pass except Exception, e: # redirect all unhandled exceptions to the parent syncstate.things_to_do = True syncstate.temp_exc = e while self.parent is not None and self.parent.frame is None: # greenlet behavior is fine self.parent = self.parent.parent return state.update(self.parent) ------------------------------------------- I've inserted the 'self.finished()' hook yesterday, to give applevel coroutines (in this case tasklets) the chance to do their last rites. Specifically, I needed that so that the stackless scheduler has a chance to stay in sync. Now, the tasklet.finished method contains a 'schedule'. What happens is that the Exception handling in _execute above is ignored. Bad. As a next test, I've moved the 'self.finished()' call right before the return. This certainly gives the exception handling a chance to do it's thing and the stackless code still does it things. But now I get the following error message from py.py: Traceback (most recent call last): File "/usr/local/bin/py.py", line 207, in ? sys.exit(main_(sys.argv)) File "/usr/local/bin/py.py", line 118, in main_ if not main.run_toplevel(space, doit, verbose=Options.verbose): File "/home/stephan/projects/pypy-dist/pypy/interpreter/main.py", line 80, in run_toplevel f() File "/usr/local/bin/py.py", line 105, in doit main.run_file(args[0], space=space) File "/home/stephan/projects/pypy-dist/pypy/interpreter/main.py", line 67, in run_file run_string(istring, filename, space) File "/home/stephan/projects/pypy-dist/pypy/interpreter/main.py", line 58, in run_string _run_eval_string(source, filename, space, False) File "/home/stephan/projects/pypy-dist/pypy/interpreter/main.py", line 47, in _run_eval_string retval = pycode.exec_code(space, w_globals, w_globals) File "/home/stephan/projects/pypy-dist/pypy/interpreter/eval.py", line 26, in exec_code return frame.run() File "/home/stephan/projects/pypy-dist/pypy/interpreter/eval.py", line 162, in resume executioncontext.leave(self) File "/home/stephan/projects/pypy-dist/pypy/interpreter/executioncontext.py", line 38, in leave self.framestack.pop() File "/home/stephan/projects/pypy-dist/pypy/interpreter/miscutils.py", line 34, in pop return self.items.pop() IndexError: pop from empty list ---------------------------------------------------- So, somehow I either have nice stackless scheduler clean up code, or I have proper exception handling. If somebody has a nice idea what to do, I'd be very happy to hear about that. Cheers Stephan From mwh at python.net Thu Jun 15 16:36:20 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 15 Jun 2006 15:36:20 +0100 Subject: [pypy-dev] 0.9 release meeting 11AM (GMT+2) References: <20060615055649.GV25313@solar.trillke> Message-ID: <2mslm6mqor.fsf@starship.python.net> hpk at trillke.net (holger krekel) writes: > Hi folks, > > today at 11AM (GMT+2) we'll meet on #pypy-release > for a meeting regarding the upcoming 0.9 release Here's an attempt at a summary of the current status and current plans for work before the release. Our original plan called for a release branch to be made tomorrow night. This now seems rather too early, and we're aiming for makint branch on Monday evening instead. The release is still planned for Thursday. If anyone between now and the 0.9 release commits code that breaks translation, it gets reverted. Please don't :) > * extension compiler The status is that it works, apart from four things: * it doesn't support modules that have an app-level component properly * you can't define "special methods" (__add__, etc) on exported types * you can't test the mixed module on top of cpython * documenation. We decided that the second of these is not that important at this stage, but the others need work. Armin is going to work on it today, at least. Armin is going to take overall responsibility for this item. > * stackless features The main problem with this item is that the status is that we don't know what the status is :( We need a way of running tests with a pypy-c and more tests to run. There are three subtopics: * tasklet cloning * tasklet pickling * exposing features to applications. A while ago I promised to play with exposed tasklet cloning to app level; various stupid mistakes on my part, a lot of distractions and the fact that stacklessgc builds take ages mean I'm not done yet, but I'm getting there. We still think tasklet pickling is working, but given the lack of tests, it's hard to be sure (see above). The exposing of app level features to applications is roughly speaking done, but needs more tests (see above). Hogler had hoped to work on running tests with pypy-c but may not be able to. Michael and him will discuss approaches and one of them will implement something by tomorrow, allowing Eric, Stephan, me, Carl and possibly some others to write tests over the weekend or next week. Michael and Samuele will coordinate this issue. > * (high level backends / snapshot of work) We'd like to have some kind of preview and documentation of a high level backend in the 0.9 release, which basically means the CLI backend. Unfornuately "Mr. CLI Backend, Antonio, has exams this and next week and so he doesn't have much time to work on this. Nevertheless, him and Armin will confer and decide on a practical approach -- some integration of the CLI backend into translatorshell.py and driver.py and a section in getting-started seem like a resonable target. Armin will coordinate this issue. > * logic/constraint works It would be nice to have a translatable logic object space in 0.9, but this is a lot of work and isn't going to happen in time. It more or less works on top of py.py though, so we'll settle for making it work properly on py.py (Carl) and writing some documentation (Anders). The documentation was promised by Tuesday night. Anders and Carl will coordinate this issue. > * feature freeze (branching off?) As discussed above, aiming for Monday night. > * documentation about all this A certain amount of discussion about documentation took place while talking about the above items, and in general the person responsible for coordinating an item is responsible for the docs for that item too. In addition, I promised to review translation.txt (which is now rather out of date) and Arre, with Armin's help, promised to document rctypes. There was an additional item: * general stability that got mentioned at the end of the meeting. We've now fixed _most_ of the problems that running CPython's test suite with pypy-c revealed, but there are issues around the area of very large (i.e. close to LONG_MAX) allocations. It's not clear what we can get done about this before 0.9, but hopefully we can clear most of them up. Cheers, mwh -- To summarise the summary of the summary:- people are a problem. -- The Hitch-Hikers Guide to the Galaxy, Episode 12 From anto.cuni at gmail.com Fri Jun 16 21:14:00 2006 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 16 Jun 2006 21:14:00 +0200 Subject: [pypy-dev] Ladies and gentlemen... Message-ID: <449302F8.8050509@gmail.com> ... I'm proud to announce that gencli is the first high level backend that can compile and run rpystone and richards :-). Some early benchmarks (I post them here so I'll know where to find them when I need it later :-)): pystone ------- gencli: 177429.668653 rpystones/second genc: 4926108.374384 rpystones/second genc w/o backendopt: 1592356.687898 rpystoned/second richards -------- gencli: 28.652590 ms/iteration genc: 7.431851 ms/iteration genc w/o backendopt: 16.203486 ms/iteration ciao Anto From arigo at tunes.org Sat Jun 17 12:15:59 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 17 Jun 2006 12:15:59 +0200 Subject: [pypy-dev] 0.9 release meeting 11AM (GMT+2) In-Reply-To: <2mslm6mqor.fsf@starship.python.net> References: <20060615055649.GV25313@solar.trillke> <2mslm6mqor.fsf@starship.python.net> Message-ID: <20060617101558.GA17735@code0.codespeak.net> Hi, Here is an update: On Thu, Jun 15, 2006 at 03:36:20PM +0100, Michael Hudson wrote: > > * extension compiler > > * it doesn't support modules that have an app-level component properly > * you can't define "special methods" (__add__, etc) on exported types > * you can't test the mixed module on top of cpython > * documenation. I finished this, apart from the "special methods" item. > Arre, with Armin's help, promised to document > rctypes. I collected the existing documentation, and removed bits that are out-of-date design documents. It's now all in pypy/doc/rctypes.txt. It could probably do with some completing or polishing. A bientot, Armin From arigo at tunes.org Sat Jun 17 12:17:45 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 17 Jun 2006 12:17:45 +0200 Subject: [pypy-dev] Ladies and gentlemen... In-Reply-To: <449302F8.8050509@gmail.com> References: <449302F8.8050509@gmail.com> Message-ID: <20060617101745.GB17735@code0.codespeak.net> Hi Anto, On Fri, Jun 16, 2006 at 09:14:00PM +0200, Antonio Cuni wrote: > ... I'm proud to announce that gencli is the first high level backend > that can compile and run rpystone and richards :-). (Applause) Great! Now translating the whole of PyPy can't be that far away any more :-) Armin From arigo at tunes.org Sun Jun 18 23:44:49 2006 From: arigo at tunes.org (Armin Rigo) Date: Sun, 18 Jun 2006 23:44:49 +0200 Subject: [pypy-dev] Random updates Message-ID: <20060618214449.GA15178@code0.codespeak.net> Hi all, I did a bit of detective work this evening, and found the following. The reason Eric's cron job is no longer showing results for stackless builds is that we broke old-stackless builds of PyPy by adding resume point operations (for unpickling tasklets). They end up as OP_RESUME_STATE_CREATE() macro calls in genc! Argh. Either with fix by somehow detecting in the _stackless module if we are being compiled with old- or new-stackless. Or we don't care any more about the old-stackless and declare it dead for the purpose of compiling PyPy. I also found out that the test summary page at http://snake.cs.uni-duesseldorf.de/pypytest/summary.html was doing a great job at hiding the fact that the tests no longer ran to the end, and that, for more than two weeks now. Either test_ee in pyllvm was getting a SIGIOT signal (people who ever heard about this signal, raise your hand), or test_wrapping was segfaulting (unreproductably - fun). I still have no clue why test_wrapping occasionally segfaults. In any case I fixed the output by considering abnormal termination as a failure. It shows up now at the beginning of the failures. I also modified the misleading '.', which previously only meant 'this test did not fail in that revision' -- e.g. because they were not run at all :-/ Now, a '.' really means 'known to have passed in that revision'. Otherwise you can get an 's', or a ' ' if the test wasn't run at all. Note that before the 12th of June, the necessary information was not collected by the run, so all non-failures there show up as ' ' even if the test was actually run. A bientot, Armin. From elmo13 at jippii.fi Mon Jun 19 00:36:54 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Mon, 19 Jun 2006 01:36:54 +0300 Subject: [pypy-dev] Random updates In-Reply-To: <20060618214449.GA15178@code0.codespeak.net> References: <20060618214449.GA15178@code0.codespeak.net> Message-ID: <4495D586.4090907@jippii.fi> Armin Rigo wrote: > ... > > I also found out that the test summary page at > http://snake.cs.uni-duesseldorf.de/pypytest/summary.html was doing a > great job at hiding the fact that the tests no longer ran to the end, > and that, for more than two weeks now. Either test_ee in pyllvm was > getting a SIGIOT signal (people who ever heard about this signal, raise > your hand), or test_wrapping was segfaulting (unreproductably - fun). > I still have no clue why test_wrapping occasionally segfaults. > ... > I reported a segfault some time ago (python2.4 specific), of which you can read in the archives, but no one seemed to be interested and I didn't want to push it. My post was about compilemodule.py, but this might very well be related since they use the same code quite the same way I think. The segfaulting stopped at some point, but if you try a revision around the time that I sent my post (11.5), you might be able to reproduce it. The segfaulting itself seemed to me as a bug in python, but since it didn't happen with 2.5 (some other error did though =), I didn't see it as so important. Maybe I should've pushed it harder =). Just say if I'm totally off the course, Elmo From mistobaan at gmail.com Mon Jun 19 18:28:48 2006 From: mistobaan at gmail.com (Fabrizio Milo) Date: Mon, 19 Jun 2006 18:28:48 +0200 Subject: [pypy-dev] Ideas for Student Proposal. Message-ID: Hello to everybody, I am an Italian student who will participate in the euro-pypy-sprint. Looking for a good idea for a proposal, I saw https://codespeak.net/issue/pypy-dev/ and many issues are interesting. like: - https://codespeak.net/issue/pypy-dev/issue87 - https://codespeak.net/issue/pypy-dev/issue111 ( or/ and ) - https://codespeak.net/issue/pypy-dev/issue205 - https://codespeak.net/issue/pypy-dev/issue224 The common subject here is "Improving test machinery". What do you think, can issue87 + issue111 be a good base for a proposal? Is it required to produce some proof-of-concept? Are there other ideas in the community not listed in that page? I am a good programmer ( I am a the end of a Computer Science Master course in Rome ) so don't let me escape from helping this wonderful project :) Fabrizio Milo From hpk at trillke.net Mon Jun 19 18:57:19 2006 From: hpk at trillke.net (holger krekel) Date: Mon, 19 Jun 2006 18:57:19 +0200 Subject: [pypy-dev] Ideas for Student Proposal. In-Reply-To: References: Message-ID: <20060619165719.GS25313@solar.trillke> Hi Fabrizio! On Mon, Jun 19, 2006 at 18:28 +0200, Fabrizio Milo wrote: > I am an Italian student who will participate in the euro-pypy-sprint. cool. btw, did you already send a note to pypy-sprint at codespeak.net? please also state your timeframe and interests there (redundantly). > Looking for a good idea for a proposal, I saw > https://codespeak.net/issue/pypy-dev/ and many issues are interesting. > > like: > - https://codespeak.net/issue/pypy-dev/issue87 > - https://codespeak.net/issue/pypy-dev/issue111 > > ( or/ and ) > - https://codespeak.net/issue/pypy-dev/issue205 > - https://codespeak.net/issue/pypy-dev/issue224 > > The common subject here is "Improving test machinery". Yes, however, issue224 is more about adding tests for pypy stackless functionality (needed _this_ week :) while the other three are about actually improving the underlying test mechanisms mostly and thus belong to py.test/py library and not primarily to PyPy (although both are involved). > What do you think, can issue87 + issue111 be a good base for a proposal? > Is it required to produce some proof-of-concept? I'd suggest that you draft at least the contents of issue87, issue111 (and bits of/integration with issue205) into a proposal under your proposed headline working title. You may bounce it to me or to py-dev at codespeak.net (the py lib / py.test developer mailing list) if you have further questions. I guess the already known sub tasks would be "adding doctest support" and "refined error reporting" for py.test ensuring suitability for PyPy testing purposes (including integrating/helping with Issue205 - the pypy-c test runs). PyPy adds quite some custom functionality on top of py.test and it is sometimes a challenge to design things flexibly enough for it. > Are there other ideas in the community not listed in that page? Related to testing there is the issue of distributing tests across many machines (using py.execnet as an ad-hoc networking mechanism if you ask me). I see this as a rather separate topic from the other three, though. But there is nothing keeping you from adding another proposal after you finished your first :) > I am a good programmer ( I am a the end of a Computer Science Master > course in Rome ) > so don't let me escape from helping this wonderful project :) We will do our best :) Summary: it's best if you look around some more, ask questions and draft a proposal and let me or better py-dev review it. cheers, holger From arigo at tunes.org Wed Jun 21 11:10:29 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 21 Jun 2006 11:10:29 +0200 Subject: [pypy-dev] Random updates In-Reply-To: <4495D586.4090907@jippii.fi> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> Message-ID: <20060621091029.GA659@code0.codespeak.net> Hi Elmo, On Mon, Jun 19, 2006 at 01:36:54AM +0300, Elmo M?ntynen wrote: > > I also found out that the test summary page at > > http://snake.cs.uni-duesseldorf.de/pypytest/summary.html was doing a > > great job at hiding the fact that the tests no longer ran to the end, > > and that, for more than two weeks now. Either test_ee in pyllvm was > > getting a SIGIOT signal (people who ever heard about this signal, raise > > your hand), or test_wrapping was segfaulting (unreproductably - fun). > > I still have no clue why test_wrapping occasionally segfaults. > > ... > > > I reported a segfault some time ago (python2.4 specific), of which you > can read in the archives, but no one seemed to be interested and I > didn't want to push it. My post was about > compilemodule.py, but this might very well be related since they use > the same code quite the same way I think. No, test_wrapping is about something different than what compilemodule.py uses. The kind of wrapping tested here is only used by external tools developed by Christian Tismer, which he meanwhile checked in as pypy/translator/rool/raymond.py. As I have no full clue about what test_wrapping does (and it uses names like "test_asd", which makes it crystal clear what the intent of the test is), I think I can live with this crash for now. Sorry for not having come back to you about the segfault you reported, I could never reproduce it myself... > The segfaulting itself seemed to me as a bug in python, but since it > didn't happen with 2.5 (some other error did though =), I didn't see it > as so important. Maybe I should've pushed it harder =). So what is the current status: does compilemodule.py _demo work for you? On several Python versions? A bientot, Armin From elmo13 at jippii.fi Thu Jun 22 01:20:54 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Thu, 22 Jun 2006 02:20:54 +0300 Subject: [pypy-dev] Random updates In-Reply-To: <20060621091029.GA659@code0.codespeak.net> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> Message-ID: <4499D456.2030706@jippii.fi> Armin Rigo wrote: > Hi Elmo, > > On Mon, Jun 19, 2006 at 01:36:54AM +0300, Elmo M?ntynen wrote: > >>> I also found out that the test summary page at >>> http://snake.cs.uni-duesseldorf.de/pypytest/summary.html was doing a >>> great job at hiding the fact that the tests no longer ran to the end, >>> and that, for more than two weeks now. Either test_ee in pyllvm was >>> getting a SIGIOT signal (people who ever heard about this signal, raise >>> your hand), or test_wrapping was segfaulting (unreproductably - fun). >>> I still have no clue why test_wrapping occasionally segfaults. >>> ... >>> >>> >> I reported a segfault some time ago (python2.4 specific), of which you >> can read in the archives, but no one seemed to be interested and I >> didn't want to push it. My post was about >> compilemodule.py, but this might very well be related since they use >> the same code quite the same way I think. >> > > No, test_wrapping is about something different than what > compilemodule.py uses. The kind of wrapping tested here is only used by > external tools developed by Christian Tismer, which he meanwhile checked > in as pypy/translator/rool/raymond.py. As I have no full clue about > what test_wrapping does (and it uses names like "test_asd", which makes > it crystal clear what the intent of the test is), I think I can live > with this crash for now. > > Sorry for not having come back to you about the segfault you reported, I > could never reproduce it myself... > > >> The segfaulting itself seemed to me as a bug in python, but since it >> didn't happen with 2.5 (some other error did though =), I didn't see it >> as so important. Maybe I should've pushed it harder =). >> > > So what is the current status: does compilemodule.py _demo work for you? > On several Python versions? > 2.3 seems to work nicely, 2.4 segfaults (?! I thought it worked again), which I'll try to reproduce with different machine (I'll report soon): ... [translation:info] with policy: pypy.objspace.cpy.ann_policy.CPyAnnotatorPolicy ... segfault and latest 2.5 works. This might just be something on my end, but any pointers on what might be wrong would be useful. A small complaint about py.test: a segfaulting test crashes py.test with EOFError and with an older rev of 2.5 compilemodule.py _demo errored and I got a pdb prompt, but py.test test_compilemodule.py reported a success nonetheless!? From hpk at trillke.net Thu Jun 22 07:49:13 2006 From: hpk at trillke.net (holger krekel) Date: Thu, 22 Jun 2006 07:49:13 +0200 Subject: [pypy-dev] Random updates In-Reply-To: <4499D456.2030706@jippii.fi> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> <4499D456.2030706@jippii.fi> Message-ID: <20060622054913.GS25313@solar.trillke> Hi Elmo, On Thu, Jun 22, 2006 at 02:20 +0300, Elmo M?ntynen wrote: > A small complaint about py.test: a segfaulting test crashes py.test with > EOFError and with an older rev of 2.5 > compilemodule.py _demo errored and I got a pdb prompt, but py.test > test_compilemodule.py reported a success nonetheless!? Maybe the test doesn't do the same test as the direct invocation of the compilemodule.py does? Either way, i added a feature issue (https://codespeak.net/issue/pypy-dev/issue233) so that py.test allows more robust test modes. Thanks for the comment! It has been an issue in the past but its severity grew with more and more tests being able to segfault. best, holger From elmo13 at jippii.fi Thu Jun 22 11:25:50 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Thu, 22 Jun 2006 12:25:50 +0300 Subject: [pypy-dev] Random updates In-Reply-To: <20060622054913.GS25313@solar.trillke> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> <4499D456.2030706@jippii.fi> <20060622054913.GS25313@solar.trillke> Message-ID: <449A621E.5060206@jippii.fi> holger krekel wrote: > Hi Elmo, > > On Thu, Jun 22, 2006 at 02:20 +0300, Elmo M?ntynen wrote: > >> A small complaint about py.test: a segfaulting test crashes py.test with >> EOFError and with an older rev of 2.5 >> compilemodule.py _demo errored and I got a pdb prompt, but py.test >> test_compilemodule.py reported a success nonetheless!? >> > > Maybe the test doesn't do the same test as the direct invocation > of the compilemodule.py does? > Actually, they seem to do the very same thing: they call compilemodule('_demo'). If they weren't doing the same thing, would the test be testing the right thing? Anyway, I happened to update my python2.5 and don't have any record of what happened, so I can't be sure about it. > Either way, i added a feature issue (https://codespeak.net/issue/pypy-dev/issue233) > so that py.test allows more robust test modes. Thanks for the comment! > It has been an issue in the past but its severity grew with more and > more tests being able to segfault. > > best, > > holger > From arigo at tunes.org Thu Jun 22 11:38:13 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 22 Jun 2006 11:38:13 +0200 Subject: [pypy-dev] Random updates In-Reply-To: <4499D456.2030706@jippii.fi> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> <4499D456.2030706@jippii.fi> Message-ID: <20060622093813.GA26980@code0.codespeak.net> Hi Elmo, On Thu, Jun 22, 2006 at 02:20:54AM +0300, Elmo M?ntynen wrote: > 2.3 seems to work nicely, 2.4 segfaults (?! I thought it worked again), > which I'll try to reproduce with different machine (I'll report soon): > ... > [translation:info] with policy: > pypy.objspace.cpy.ann_policy.CPyAnnotatorPolicy > ... > segfault > and latest 2.5 works. We'd need some much more precise info about this to be able to do anything. Which version of CPython is that precisely, which PyPy revision, which OS? Can you also please paste the whole output before the segfault? If you manage to get a error and a pdb prompt with another Python version, can you also please paste the whole output here, together with the exact CPython version number? Thanks! Armin From elmo13 at jippii.fi Thu Jun 22 11:44:56 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Thu, 22 Jun 2006 12:44:56 +0300 Subject: [pypy-dev] Random updates In-Reply-To: <449A621E.5060206@jippii.fi> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> <4499D456.2030706@jippii.fi> <20060622054913.GS25313@solar.trillke> <449A621E.5060206@jippii.fi> Message-ID: <449A6698.6030109@jippii.fi> Elmo M?ntynen wrote: > holger krekel wrote: > >> Hi Elmo, >> >> On Thu, Jun 22, 2006 at 02:20 +0300, Elmo M?ntynen wrote: >> >> >>> A small complaint about py.test: a segfaulting test crashes py.test with >>> EOFError and with an older rev of 2.5 >>> compilemodule.py _demo errored and I got a pdb prompt, but py.test >>> test_compilemodule.py reported a success nonetheless!? >>> >>> >> Maybe the test doesn't do the same test as the direct invocation >> of the compilemodule.py does? >> >> > Actually, they seem to do the very same thing: they call > compilemodule('_demo'). If they weren't doing the same thing, would the > test be testing the right thing? Anyway, I happened to update my > python2.5 and don't have any record of what happened, so I can't be sure > about it. > Actually, there's one difference: try: driver.proceed(['compile_c']) except SystemExit: raise except: if not interactive: raise debug(driver) raise SystemExit(1) return driver.cbuilder.c_ext_module where interactive is True for the direct invocation. Could this just be the fault of a broken CPython interpreter? From elmo13 at jippii.fi Thu Jun 22 12:38:26 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Thu, 22 Jun 2006 13:38:26 +0300 Subject: [pypy-dev] Random updates In-Reply-To: <20060622093813.GA26980@code0.codespeak.net> References: <20060618214449.GA15178@code0.codespeak.net> <4495D586.4090907@jippii.fi> <20060621091029.GA659@code0.codespeak.net> <4499D456.2030706@jippii.fi> <20060622093813.GA26980@code0.codespeak.net> Message-ID: <449A7322.2080700@jippii.fi> Armin Rigo wrote: > Hi Elmo, > On Thu, Jun 22, 2006 at 02:20:54AM +0300, Elmo M?ntynen wrote: > >> 2.3 seems to work nicely, 2.4 segfaults (?! I thought it worked again), >> which I'll try to reproduce with different machine (I'll report soon): >> [cbuild:execute] cc -O2 -pthread -I/usr/include/python2.4 -c ctypesplatcheck_0.c -o ctypesplatcheck_0.o [cbuild:execute] cc -pthread /tmp/usession-20/ctypesplatcheck_0.o -lm -lpthread -o /tmp/usession-20/ctypesplatcheck_0 [cbuild:execute] cc -O2 -pthread -c ctypesplatcheck_1.c -o ctypesplatcheck_1.o [cbuild:execute] cc -pthread /tmp/usession-20/ctypesplatcheck_1.o -lm -lpthread -o /tmp/usession-20/ctypesplatcheck_1 [translation:info] Annotating&simplifying... [translation:info] with policy: pypy.objspace.cpy.ann_policy.CPyAnnotatorPolicy Segmentation fault > > We'd need some much more precise info about this to be able to do > anything. Which version of CPython is that precisely, which PyPy > revision, which OS? Can you also please paste the whole output before > the segfault? > CPython: Python 2.4.4c0 (#2, Jun 14 2006, 22:35:41) [GCC 4.1.2 20060613 (prerelease) (Debian 4.1.1-4)] on linux2 OS: Debian Etch, linux 2.6.15 Pypy: latest revision to be continued... > If you manage to get a error and a pdb prompt with another Python > version, can you also please paste the whole output here, together with > the exact CPython version number? > When I tried debugging this, I had hunted down the particular place where the segfault happens, and there was nothing, meaning that there wasn't anything particular that seemed like something that CPython wasn't made to handle, just some normal looking code. The gist of all this is that it seemed to me that the segfault is a plain bug in CPython, just a more obscure one that shows up with the wuite massive pypy machinery. From mwh at python.net Sun Jun 25 14:05:44 2006 From: mwh at python.net (Michael Hudson) Date: Sun, 25 Jun 2006 13:05:44 +0100 Subject: [pypy-dev] pypy-0.9.0: stackless, new extension compiler Message-ID: <2m4py9jv8n.fsf@starship.python.net> The PyPy development team has been busy working and we've now packaged our latest improvements, completed work and new experiments as version 0.9.0, our fourth public release. The highlights of this fourth release of PyPy are: **implementation of "stackless" features** We now support the larger part of the interface of the original Stackless Python -- see http://www.stackless.com for more. A significant part of this is the pickling and unpickling of a running tasklet. These features, especially the pickling, can be considered to be a "technology preview" -- they work, but for example the error handling is a little patchy in places. **ext-compiler** The "extension compiler" is a new way of writing a C extension for CPython and PyPy at the same time. For more information, see its documentation: http://codespeak.net/pypy/dist/pypy/doc/extcompiler.html **rctypes** Most useful in combination with the ext-compiler is the fact that our translation framework can translate code that uses the standard-in-Python-2.5 ctypes module. See its documentation for more: http://codespeak.net/pypy/dist/pypy/doc/rctypes.html **framework GCs** PyPy's interpreter can now be compiled to use a garbage collector written in RPython. This added control over PyPy's execution makes the implementation of new and interesting features possible, apart from being a significant achievement in its own right. **__del__/weakref/__subclasses__** The PyPy interpreter's compatibility with CPython continues improves: now we support __del__ methods, the __subclasses__ method on types and weak references. We now pass around 95% of CPython's core tests. **logic space preview** This release contains the first version of the logic object space, which will add logical variables to Python. See its docs for more: http://codespeak.net/pypy/dist/pypy/doc/howto-logicobjspace-0.9.html **high level backends preview** This release contains the first versions of new backends targeting high level languages such as Squeak and .NET/CLI and updated versions of the JavaScript and Common Lisp backends. They can't compile the PyPy interpreter yet, but they're getting there... **bugfixes, better performance** As you would expect, performance continues to improve and bugs continue to be fixed. The performance of the translated PyPy interpreter is 2.5-3x times faster than 0.8 (on richards and pystone), and is now stable enough to be able to run CPython's test suite to the end. **testing refinements** py.test, our testing tool, now has preliminary support for doctests. We now run all our tests every night, and you can see the summary at: http://snake.cs.uni-duesseldorf.de/pypytest/summary.html What is PyPy (about)? ------------------------------------------------ PyPy is a MIT-licensed research-oriented reimplementation of Python written in Python itself, flexible and easy to experiment with. It translates itself to lower level languages. Our goals are to target a large variety of platforms, small and large, by providing a compilation toolsuite that can produce custom Python versions. Platform, memory and threading models are to become aspects of the translation process - as opposed to encoding low level details into the language implementation itself. Eventually, dynamic optimization techniques - implemented as another translation aspect - should become robust against language changes. Note that PyPy is mainly a research and development project and does not by itself focus on getting a production-ready Python implementation although we do hope and expect it to become a viable contender in that area sometime next year. PyPy is partially funded as a research project under the European Union's IST programme. Where to start? ----------------------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ The interpreter and object model implementations shipped with the 0.9 version can run on their own and implement the core language features of Python as of CPython 2.4. However, we still do not recommend using PyPy for anything else than for education, playing or research purposes. Ongoing work and near term goals --------------------------------- The Just-in-Time compiler and other performance improvements will be one of the main topics of the next few months' work, along with finishing the logic object space. Project Details --------------- PyPy has been developed during approximately 20 coding sprints across Europe and the US. It continues to be a very dynamically and incrementally evolving project with many of these one-week workshops to follow. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support from numerous people. Please feel free to give feedback and raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html have fun, the pypy team, (Armin Rigo, Samuele Pedroni, Holger Krekel, Christian Tismer, Carl Friedrich Bolz, Michael Hudson, and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a two year European Union IST research grant. The full partners of that consortium are: Heinrich-Heine University (Germany), AB Strakt (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany) -- Hiro dicks about with his computer, naturally. Being stranded on a life raft in the Pacific is a perfect venue for a hacker. -- Snow Crash, Neal Stephenson From ms at cerenity.org Sun Jun 25 14:16:18 2006 From: ms at cerenity.org (Michael) Date: Sun, 25 Jun 2006 13:16:18 +0100 Subject: [pypy-dev] pypy-0.9.0: stackless, new extension compiler In-Reply-To: <2m4py9jv8n.fsf@starship.python.net> References: <2m4py9jv8n.fsf@starship.python.net> Message-ID: <200606251316.18134.ms@cerenity.org> Woooooo! Michael From Ben.Young at risk.sungard.com Mon Jun 26 11:34:08 2006 From: Ben.Young at risk.sungard.com (Ben.Young at risk.sungard.com) Date: Mon, 26 Jun 2006 10:34:08 +0100 Subject: [pypy-dev] [Python-Dev] pypy-0.9.0: stackless, new extension compiler In-Reply-To: <2mslltigm2.fsf@starship.python.net> Message-ID: Congratulations! python-dev-bounces+python=theyoungfamily.co.uk at python.org wrote on 25/06/2006 13:07:01: > The PyPy development team has been busy working and we've now packaged > our latest improvements, completed work and new experiments as > version 0.9.0, our fourth public release. > > The highlights of this fourth release of PyPy are: > > **implementation of "stackless" features** > We now support the larger part of the interface of the original > Stackless Python -- see http://www.stackless.com for more. A > significant part of this is the pickling and unpickling of a running > tasklet. > > These features, especially the pickling, can be considered to be a > "technology preview" -- they work, but for example the error handling > is a little patchy in places. > > **ext-compiler** > The "extension compiler" is a new way of writing a C extension for > CPython and PyPy at the same time. For more information, see its > documentation: http://codespeak.net/pypy/dist/pypy/doc/extcompiler.html > > **rctypes** > Most useful in combination with the ext-compiler is the fact that our > translation framework can translate code that uses the > standard-in-Python-2.5 ctypes module. See its documentation for more: > http://codespeak.net/pypy/dist/pypy/doc/rctypes.html > > **framework GCs** > PyPy's interpreter can now be compiled to use a garbage collector > written in RPython. This added control over PyPy's execution makes the > implementation of new and interesting features possible, apart from > being a significant achievement in its own right. > > **__del__/weakref/__subclasses__** > The PyPy interpreter's compatibility with CPython continues improves: > now we support __del__ methods, the __subclasses__ method on types and > weak references. We now pass around 95% of CPython's core tests. > > **logic space preview** > This release contains the first version of the logic object space, > which will add logical variables to Python. See its docs for more: > http://codespeak.net/pypy/dist/pypy/doc/howto-logicobjspace-0.9.html > > **high level backends preview** > This release contains the first versions of new backends targeting high > level languages such as Squeak and .NET/CLI and updated versions of the > JavaScript and Common Lisp backends. They can't compile the PyPy > interpreter yet, but they're getting there... > > **bugfixes, better performance** > As you would expect, performance continues to improve and bugs continue > to be fixed. The performance of the translated PyPy interpreter is > 2.5-3x times faster than 0.8 (on richards and pystone), and is now > stable enough to be able to run CPython's test suite to the end. > > **testing refinements** > py.test, our testing tool, now has preliminary support for doctests. > We now run all our tests every night, and you can see the summary at: > http://snake.cs.uni-duesseldorf.de/pypytest/summary.html > > What is PyPy (about)? > ------------------------------------------------ > > PyPy is a MIT-licensed research-oriented reimplementation of Python > written in Python itself, flexible and easy to experiment with. It > translates itself to lower level languages. Our goals are to target a > large variety of platforms, small and large, by providing a > compilation toolsuite that can produce custom Python versions. > Platform, memory and threading models are to become aspects of the > translation process - as opposed to encoding low level details into > the language implementation itself. Eventually, dynamic optimization > techniques - implemented as another translation aspect - should become > robust against language changes. > > Note that PyPy is mainly a research and development project and does > not by itself focus on getting a production-ready Python > implementation although we do hope and expect it to become a viable > contender in that area sometime next year. > > PyPy is partially funded as a research project under the European > Union's IST programme. > > Where to start? > ----------------------------- > > Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting- > started.html > > PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ > > PyPy Homepage: http://codespeak.net/pypy/ > > The interpreter and object model implementations shipped with the 0.9 > version can run on their own and implement the core language features > of Python as of CPython 2.4. However, we still do not recommend using > PyPy for anything else than for education, playing or research > purposes. > > Ongoing work and near term goals > --------------------------------- > > The Just-in-Time compiler and other performance improvements will be one of > the main topics of the next few months' work, along with finishing the > logic object space. > > Project Details > --------------- > > PyPy has been developed during approximately 20 coding sprints across > Europe and the US. It continues to be a very dynamically and > incrementally evolving project with many of these one-week workshops > to follow. > > PyPy has been a community effort from the start and it would > not have got that far without the coding and feedback support > from numerous people. Please feel free to give feedback and > raise questions. > > contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html > > have fun, > > the pypy team, (Armin Rigo, Samuele Pedroni, > Holger Krekel, Christian Tismer, > Carl Friedrich Bolz, Michael Hudson, > and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) > > PyPy development and activities happen as an open source project > and with the support of a consortium partially funded by a two > year European Union IST research grant. The full partners of that > consortium are: > > Heinrich-Heine University (Germany), AB Strakt (Sweden) > merlinux GmbH (Germany), tismerysoft GmbH (Germany) > Logilab Paris (France), DFKI GmbH (Germany) > ChangeMaker (Sweden), Impara (Germany) > > -- > And not only in the sense that they imagine heretics where these > do not exist, but also that inquistors repress the heretical > putrefaction so vehemently that many are driven to share in it, > in their hatred of the judges. -- The Name Of The Rose, Umberto Eco > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/python%40theyoungfamily.co.uk > From arigo at tunes.org Tue Jun 27 10:34:41 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 27 Jun 2006 10:34:41 +0200 Subject: [pypy-dev] pypy-0.9.0: stackless, new extension compiler Message-ID: <20060627083441.GA485@code0.codespeak.net> Hi all, (message by cfbolz which I'm forwarding here) _______________________________________________ Unfortunately the download links for the release tarballs did not work until very recently. They are now working though. You can download the 0.9 release of PyPy under: http://codespeak.net/download/pypy/pypy-0.9.0.tar.bz2 http://codespeak.net/download/pypy/pypy-0.9.0.tar.gz http://codespeak.net/download/pypy/pypy-0.9.0.zip For detailed notes about how to get started into the world of PyPy see here: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html Sorry for the fuss and cheers, Carl Friedrich Bolz _______________________________________________ From cfbolz at gmx.de Tue Jun 27 10:50:54 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 27 Jun 2006 10:50:54 +0200 Subject: [pypy-dev] pypy-0.9.0: stackless, new extension compiler In-Reply-To: <20060627083441.GA485@code0.codespeak.net> References: <20060627083441.GA485@code0.codespeak.net> Message-ID: <348899050606270150s24e15821vb8c88e69b1a7556f@mail.gmail.com> Hi Armin! 2006/6/27, Armin Rigo : > Hi all, (message by cfbolz which I'm forwarding here) wuaa, thanks. I sent the mail only to Michael :-) Cheers, Carl Friedrich From wanja at merlinux.de Thu Jun 29 11:58:00 2006 From: wanja at merlinux.de (Wanja Saatkamp) Date: Thu, 29 Jun 2006 11:58:00 +0200 Subject: [pypy-dev] Release of PyPy videos Message-ID: <44A3A428.60408@merlinux.de> The PyPy team is happy to announce that the first bunch of PyPy videos can now be downloaded from: http://codespeak.net/pypy/dist/pypy/doc/video-index.html The videos introduce involved people and contain different talks, tutorials and interviews such as: 1. Technical talk on the PyPy architecture, standard interpreter, translation toolchain and Just-in-time compiler at the University of Palma de Mallorca (72 min) 2. Coding discussion between Armin Rigo and Samuele Pedroni during the PyPy sprint at the University of Palma de Mallorca (40 min) 3. Core developer Holger Krekel and project manager Beatrice During explain the agile open source methods used in the PyPy project at Pycon, Dallas (26 min) 4. Core developers Michael Hudson and Christian Tismer are giving an introductory talk at Pycon, Dallas (28 min) 5. A PyPy architecture session given by core developers Holger Krekel and Armin Rigo at Pycon, Dallas (48 min) 6. A Sprint tutorial by core developer Michael Hudson who provides a detailed and hands-on overview about the architecture of PyPy, especially the translation toolchain (44 min) 7. What do you think about PyPy? Interview with American software developer Bob Ippolito at this year's PyCon, Dallas (8 min) 8. Interview with CPython core developer Tim Peters at this year's PyCon, Dallas (23 min) Some technical details: All films are in European PAL system, encoded with DivX and packed in .avi file format. The files can be downloaded using a bittorrent client. Further details regarding downloading and viewing can be found at the URL above. Have fun and don't hesitate to contact us at pypy-dev at codespeak.net if you encounter any problems or have any other comments, Best regards, Wanja Saatkamp & the PyPy team From pedronis at strakt.com Fri Jun 30 19:44:34 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Fri, 30 Jun 2006 19:44:34 +0200 Subject: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test In-Reply-To: <20060630140821.989F310081@code0.codespeak.net> References: <20060630140821.989F310081@code0.codespeak.net> Message-ID: <44A56302.2090808@strakt.com> antocuni at codespeak.net wrote: > Author: antocuni > Date: Fri Jun 30 16:08:16 2006 > New Revision: 29534 > > Modified: > pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > Log: > Added a failing test. > > Basically, if we modify the Record after its hash has already been > computed, we might obtain two objects that compare equal but hash > differently. > > Disabling the caching in LowLevelType.__hash__ fix the test, but I > think it's not what we want. > > Adding the following __hash__ to ootype.Record also fix the test, but > I'm not sure if it's correct, especially with recursive structures: > > def __hash__(self): > return hash(self._fields) > Records are really like lltypesystem Structs, although because some of their code was copied from Instance, they may give the impression that you can add fields after the fact, but that should not be done, it breaks the fact that they are supposed to compare by structure. If you are needing that you need some other approach or introduce some kind of forward definition. _add_fields should really disappear from Record. > > > > Modified: pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py > ============================================================================== > --- pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py (original) > +++ pypy/dist/pypy/rpython/ootypesystem/test/test_oorecord.py Fri Jun 30 16:08:16 2006 > @@ -38,3 +38,14 @@ > t.a = 1 > t.b = 2 > assert ooidentityhash(t) != ooidentityhash(t2) > + > +import py > +def test_hash(): > + #py.test.skip("LowLevelType.__hash__ bug waiting to be fixed") > + T1 = Record({"item0": Signed, "item1": Signed}) > + T2 = Record({"item0": Signed}) > + > + hash(T2) # compute the hash, it will stored in __cached_hash > + T2._add_fields({"item1": Signed}) # modify the object > + assert T1 == T2 > + assert hash(T1) == hash(T2) > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn From anto.cuni at gmail.com Fri Jun 30 20:28:22 2006 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 30 Jun 2006 20:28:22 +0200 Subject: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test In-Reply-To: <44A56302.2090808@strakt.com> References: <20060630140821.989F310081@code0.codespeak.net> <44A56302.2090808@strakt.com> Message-ID: <44A56D46.7040605@gmail.com> Samuele Pedroni wrote: > Records are really like lltypesystem Structs, although because some of > their code was copied from Instance, they may give the impression > that you can add fields after the fact, but that should not be done, > it breaks the fact that they are supposed to compare by structure. > > If you are needing that you need some other approach or introduce > some kind of forward definition. > > _add_fields should really disappear from Record. ok, so it's my fault, but it don't solve my original problem. The problem is that TestCliTuple.test_inst_tuple_add_getitem in test_rpython.py used to fail because the IL code contained two copies of the same Record. After a bit of investigation I discovered that the reason was because I got two Records that compare equal but have different hashes; again, the problem was that the __cached_hash was different from the real one. Then I tried to reproduce the bug, and so I wrote the failing test, thinking that using _add_fields was fine. But after your comment I've understood that this is not the point, because Record._add_fields is called only in its __init__. For now the problem is worked-around in database.py (the lines marked with "XXX: temporary hack"), but the bug is still here: any idea of why I get two equal Records with different hashes? ciao Anto From anto.cuni at gmail.com Fri Jun 30 20:54:18 2006 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 30 Jun 2006 20:54:18 +0200 Subject: [pypy-dev] [pypy-svn] r29534 - pypy/dist/pypy/rpython/ootypesystem/test In-Reply-To: <44A56302.2090808@strakt.com> References: <20060630140821.989F310081@code0.codespeak.net> <44A56302.2090808@strakt.com> Message-ID: <44A5735A.6090908@gmail.com> Ok, I've found the bug, it's entirely a my fault, because in translator/cli/record.py I attach a new attribute _name to the Record, so the hash is no longer valid. Probably it's safer to store the name in some dictionary. ciao Anto