From tvoglou at gmail.com Tue Feb 1 09:01:26 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Tue, 1 Feb 2011 10:01:26 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit Message-ID: Amaury, It seems that there is a general issue with the compiler/link . I did the translations without the _hashlib and ssl and after a while I got the following errors. [platform:ERROR] Creating library c:\users\flatline\appdata\local\temp\usessi on-default-57\testing_1\libpypy-c.lib and object c:\users\flatline\appdata\local\temp\usession-default-57\testing_1\libpypy-c.exp [platform:ERROR] implement_52.obj : error LNK2019: unresolved external symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr [platform:ERROR] implement_64.obj : error LNK2019: unresolved external symbol _inflateInit2_ referenced in function _pypy_g_ccall_inflateInit2___z_streamPtr_Sig ned_arrayPtr [platform:ERROR] implement_64.obj : error LNK2019: unresolved external symbol _inflate referenced in function _pypy_g_ccall_inflate__z_streamPtr_Signed [platform:ERROR] implement_67.obj : error LNK2019: unresolved external symbol _deflateEnd referenced in function _pypy_g_ccall_deflateEnd__z_streamPtr [platform:ERROR] implement_69.obj : error LNK2019: unresolved external symbol _deflate referenced in function _pypy_g_ccall_deflate__z_streamPtr_Signed [platform:ERROR] implement_70.obj : error LNK2019: unresolved external symbol _adler32 referenced in function _pypy_g_ccall_adler32__Unsigned_arrayPtr_Unsigned [platform:ERROR] implement_71.obj : error LNK2019: unresolved external symbol _deflateInit2_ referenced in function _pypy_g_ccall_deflateInit2___z_streamPtr_Sig ned_Signed_S [platform:ERROR] implement_71.obj : error LNK2019: unresolved external symbol _crc32 referenced in function _pypy_g_ccall_crc32__Unsigned_arrayPtr_Unsigned [platform:ERROR] c:\users\flatline\appdata\local\temp\usession-default-57\testing_1\libpypy-c.dll : fatal error LNK1120: 8 unresolved externals [Timer] Timings: [Timer] annotate --- 1447.8 s [Timer] rtype_lltype --- 768.2 s [Timer] backendopt_lltype --- 415.9 s [Timer] stackcheckinsertion_lltype --- 60.1 s [Timer] database_c --- 498.3 s [Timer] source_c --- 1190.5 s [Timer] compile_c --- 216.4 s [Timer] =========================================== [Timer] Total: --- 4597.3 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 290, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\driver.py", line 812, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\tool\taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\driver.py", line 288, in _do [translation:ERROR] res = func() [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\driver.py", line 575, in task_compile_c [translation:ERROR] cbuilder.compile(**kwds) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py", line 527, in compile [translation:ERROR] self.executable_name = compiler.build(shared=shared) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py", line 95, in build [translation:ERROR] return self._build(shared=shared) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\c\genc.py", line 90, in _build [translation:ERROR] standalone=not shared) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py", line 52, in compile [translation:ERROR] return self._finish_linking(ofiles, eci, outputfilename, standalone) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py", line 194, in _finish_linking [translation:ERROR] return self._link(cc_link, ofiles, largs, standalone, exe_name) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\platform\windows.py", line 176, in _link [translation:ERROR] self._execute_c_compiler(self.link, args, exe_name) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\platform\__init__.py", line 118, in _execute_c_compiler [translation:ERROR] self._handle_error(returncode, stderr, stdout, outname) [translation:ERROR] File "c:\Users\flatline\development\pypy\src\pypy\translator\platform\windows.py", line 199, in _handle_error [translation:ERROR] raise CompilationError(stdout, stderr) [translation:ERROR] CompilationError: Thanks, Tasos On Mon, Jan 31, 2011 at 11:48 PM, Tasos Vogiatzoglou wrote: > Amaury, > > Isn't there a way to fix the SSL issue? I was looking checking for > name decoration issues or call-conv issues but there seems none. > > Is there any guess why it fails from the translation procedure but > works when I execute the commands in cmd prompt ? > > (I also tried disabling the env processing that is happening, leaving > POpen to get the default one but nothing :( ) > > Thanks, > Tasos > > > > On Mon, Jan 31, 2011 at 11:44 PM, Amaury Forgeot d'Arc > wrote: >> 2011/1/31 Tasos Vogiatzoglou : >>> Hello Amaury, >>> >>> Yes. It's the same compiler error : >> >>> [translation:ERROR] ? ?File "c:\Users\flatline\development\pypy\src\pypy\module\_hashlib\interp_hashlib.py", >> >> Ah, indeed the _hashlib module should be disabled when openssl cannot >> be processed. >> I'll try to fix it, in the meantime I suggest you to skip this module: >> >> python translate.py targetpypystandalone --withoutmod-_hashlib >> >> >> -- >> Amaury Forgeot d'Arc >> > From amauryfa at gmail.com Tue Feb 1 09:11:04 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 1 Feb 2011 09:11:04 +0100 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: Message-ID: Hi, 2011/2/1 Tasos Vogiatzoglou : > Amaury, > > It seems that there is a general issue with the compiler/link . > > I did the translations without the _hashlib and ssl and after a while > I got the following errors. [...] > [platform:ERROR] implement_52.obj : error LNK2019: unresolved external > symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr We have never tried the win64 platform, and I don't have access to a Windows 64bit machine. But I suspect that even if you removed all external dependencies, the result would not work; pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) Before running a translation, could you run the tests in pypy/translator/c ? in the pypy directory, run: python test_all.py translator/c I'd like at least the files "test_genc" and "test_newgc" to pass without errors. -- Amaury Forgeot d'Arc From fijall at gmail.com Tue Feb 1 09:14:19 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 1 Feb 2011 10:14:19 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 10:11 AM, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/2/1 Tasos Vogiatzoglou : >> Amaury, >> >> It seems that there is a general issue with the compiler/link . >> >> I did the translations without the _hashlib and ssl and after a while >> I got the following errors. > [...] >> [platform:ERROR] implement_52.obj : error LNK2019: unresolved external >> symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr > > We have never tried the win64 platform, and I don't have access to a > Windows 64bit machine. > But I suspect that even if you removed all external dependencies, the > result would not work; > pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) > > Before running a translation, could you run the tests in pypy/translator/c ? > in the pypy directory, run: > ? ?python test_all.py translator/c > I'd like at least the files "test_genc" and "test_newgc" to pass without errors. Surprisingly enough, I have > > -- > Amaury Forgeot d'Arc > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From tvoglou at gmail.com Tue Feb 1 09:20:55 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Tue, 1 Feb 2011 10:20:55 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: Message-ID: <-3229585076398878208@unknownmsgid> Amaury, I'll try and I'll let you know. Thanks, Tasos On 1 ??? 2011, at 10:11, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/2/1 Tasos Vogiatzoglou : >> Amaury, >> >> It seems that there is a general issue with the compiler/link . >> >> I did the translations without the _hashlib and ssl and after a while >> I got the following errors. > [...] >> [platform:ERROR] implement_52.obj : error LNK2019: unresolved external >> symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr > > We have never tried the win64 platform, and I don't have access to a > Windows 64bit machine. > But I suspect that even if you removed all external dependencies, the > result would not work; > pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) > > Before running a translation, could you run the tests in pypy/translator/c ? > in the pypy directory, run: > python test_all.py translator/c > I'd like at least the files "test_genc" and "test_newgc" to pass without errors. > > -- > Amaury Forgeot d'Arc From fijall at gmail.com Tue Feb 1 10:47:24 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 1 Feb 2011 11:47:24 +0200 Subject: [pypy-dev] Worrying set of checkins Message-ID: Anyone feeling like looking into this: http://speed.pypy.org/changes/?tre=10&rev=41510:77f94f44989a&exe=3&env=tannit ? From amauryfa at gmail.com Tue Feb 1 11:17:14 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Tue, 1 Feb 2011 11:17:14 +0100 Subject: [pypy-dev] Worrying set of checkins In-Reply-To: References: Message-ID: 2011/2/1 Maciej Fijalkowski : > Anyone feeling like looking into this: > > http://speed.pypy.org/changes/?tre=10&rev=41510:77f94f44989a&exe=3&env=tannit This show benchmarks for pypy *without* JIT. the pypy-c-jit figures are more positive. -- Amaury Forgeot d'Arc From fijall at gmail.com Tue Feb 1 11:19:57 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 1 Feb 2011 12:19:57 +0200 Subject: [pypy-dev] Worrying set of checkins In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 12:17 PM, Amaury Forgeot d'Arc wrote: > 2011/2/1 Maciej Fijalkowski : >> Anyone feeling like looking into this: >> >> http://speed.pypy.org/changes/?tre=10&rev=41510:77f94f44989a&exe=3&env=tannit > > This show benchmarks for pypy *without* JIT. > the pypy-c-jit figures are more positive. > > -- > Amaury Forgeot d'Arc > Yes. Still, worth looking into. From fijall at gmail.com Tue Feb 1 11:24:15 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Tue, 1 Feb 2011 12:24:15 +0200 Subject: [pypy-dev] Worrying set of checkins In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 12:19 PM, Maciej Fijalkowski wrote: > On Tue, Feb 1, 2011 at 12:17 PM, Amaury Forgeot d'Arc > wrote: >> 2011/2/1 Maciej Fijalkowski : >>> Anyone feeling like looking into this: >>> >>> http://speed.pypy.org/changes/?tre=10&rev=41510:77f94f44989a&exe=3&env=tannit >> >> This show benchmarks for pypy *without* JIT. >> the pypy-c-jit figures are more positive. >> >> -- >> Amaury Forgeot d'Arc >> > > Yes. Still, worth looking into. > Well, false alarms. Next run nulled out those changes. From tvoglou at gmail.com Tue Feb 1 18:10:42 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Tue, 1 Feb 2011 19:10:42 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: <-3229585076398878208@unknownmsgid> References: <-3229585076398878208@unknownmsgid> Message-ID: Amaury, You were right, there are some issues with type sizes. test_genc and test_newgc are failing. I'll try to see what I can do to fix them. Are there any directions I should move towards ? Thanks, Tasos On Tue, Feb 1, 2011 at 10:20 AM, Tasos Vogiatzoglou wrote: > Amaury, > > I'll try and I'll let you know. > > Thanks, > Tasos > > On 1 ??? 2011, at 10:11, Amaury Forgeot d'Arc wrote: > >> Hi, >> >> 2011/2/1 Tasos Vogiatzoglou : >>> Amaury, >>> >>> It seems that there is a general issue with the compiler/link . >>> >>> I did the translations without the _hashlib and ssl and after a while >>> I got the following errors. >> [...] >>> [platform:ERROR] implement_52.obj : error LNK2019: unresolved external >>> symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr >> >> We have never tried the win64 platform, and I don't have access to a >> Windows 64bit machine. >> But I suspect that even if you removed all external dependencies, the >> result would not work; >> pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) >> >> Before running a translation, could you run the tests in pypy/translator/c ? >> in the pypy directory, run: >> ? ?python test_all.py translator/c >> I'd like at least the files "test_genc" and "test_newgc" to pass without errors. >> >> -- >> Amaury Forgeot d'Arc > From renesd at gmail.com Tue Feb 1 18:32:36 2011 From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=) Date: Tue, 1 Feb 2011 17:32:36 +0000 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: Hi, A naive guess would be the different sizes between 64bit windows, and 64 bit linux? (long int is 32bit on windows and 64bit on linux) >From http://en.wikipedia.org/wiki/64-bit 64-bit data models Data model short (integer) int long (integer) long long pointers/size_t Sample operating systems LLP64/ IL32P64 16 32 32 64 64 Microsoft Windows (X64/IA-64) LP64/ I32LP64 16 32 64 64 64 Most Unix and Unix-like systems, e.g. Solaris, Linux, and Mac OS X ILP64 16 64 64 64 64 HAL Computer Systems port of Solaris to SPARC64 SILP64 64 64 64 64 64 Unicos On Tue, Feb 1, 2011 at 5:10 PM, Tasos Vogiatzoglou wrote: > Amaury, > > You were right, there are some issues with type sizes. test_genc and > test_newgc are failing. > > I'll try to see what I can do to fix them. Are there any directions I > should move towards ? > > Thanks, > Tasos > > > > On Tue, Feb 1, 2011 at 10:20 AM, Tasos Vogiatzoglou wrote: >> Amaury, >> >> I'll try and I'll let you know. >> >> Thanks, >> Tasos >> >> On 1 ??? 2011, at 10:11, Amaury Forgeot d'Arc wrote: >> >>> Hi, >>> >>> 2011/2/1 Tasos Vogiatzoglou : >>>> Amaury, >>>> >>>> It seems that there is a general issue with the compiler/link . >>>> >>>> I did the translations without the _hashlib and ssl and after a while >>>> I got the following errors. >>> [...] >>>> [platform:ERROR] implement_52.obj : error LNK2019: unresolved external >>>> symbol _inflateEnd referenced in function _pypy_g_ccall_inflateEnd__z_streamPtr >>> >>> We have never tried the win64 platform, and I don't have access to a >>> Windows 64bit machine. >>> But I suspect that even if you removed all external dependencies, the >>> result would not work; >>> pypy's compilation tools implicitly assume that sizeof(long)==sizeof(void*) >>> >>> Before running a translation, could you run the tests in pypy/translator/c ? >>> in the pypy directory, run: >>> ? ?python test_all.py translator/c >>> I'd like at least the files "test_genc" and "test_newgc" to pass without errors. >>> >>> -- >>> Amaury Forgeot d'Arc >> > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From pgiarrusso at mathematik.uni-marburg.de Wed Feb 2 00:35:54 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Wed, 2 Feb 2011 00:35:54 +0100 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: 2011/2/1 Ren? Dudfield : > Hi, > > A naive guess would be the different sizes between 64bit windows, and > 64 bit linux? ?(long int is 32bit on windows and 64bit on linux) > > From http://en.wikipedia.org/wiki/64-bit > > 64-bit data models > Data model ? ? ?short (integer) int ? ? long (integer) ?long > long ? ?pointers/size_t Sample operating systems > LLP64/ > IL32P64 16 ? ? ?32 ? ? ?32 ? ? ?64 ? ? ?64 ? ? ?Microsoft Windows (X64/IA-64) > LP64/ > I32LP64 16 ? ? ?32 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?Most Unix and Unix-like systems, e.g. Solaris, > Linux, and Mac OS X > ILP64 ? 16 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?HAL Computer Systems port of Solaris to SPARC64 > SILP64 ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?Unicos >From what Amaury said, you need to identify places where long is used and it is assumed that it has 64bits. A first guess, in general, is that _every_ use of long assumes it to be 64bit on 64bits systems. What about using intptr_t (or uintptr_t) from stdint.h? IOW, you'd simply replace long by this type everywhere, including in functions which generate C code through some library. I don't know the details, could somebody help here? IIRC this header was introduced in C99, and for sure this type is optional - i.e., it must be declared if it is available at all. Of course, there might be systems where this type is not available, so you need to provide a fallback (in a configure-like fashion). If this header is available on (modern) Windows environments (the only system where long is 32 bits), using "long" as fallback is safe. Cheers, -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From lac at openend.se Wed Feb 2 00:51:36 2011 From: lac at openend.se (Laura Creighton) Date: Wed, 02 Feb 2011 00:51:36 +0100 Subject: [pypy-dev] testing floating point Message-ID: <201102012351.p11NpaJk011803@theraft.openend.se> running the test suite of this package might be useful. Laura ------- Forwarded Message Return-Path: python-announce-list-bounces+lac=openend.se at python.org Delivery-Date: Wed Feb 2 00:29:19 2011 Return-Path: Received: from mail.python.org (mail.python.org [82.94.164.166]) by theraft.openend.se (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id p11NTCgZ009584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 2 Feb 2011 00:29:14 +0100 Received: from albatross.python.org (localhost [127.0.0.1]) by mail.python.org (Postfix) with ESMTP id 1089FEE9C0 for ; Wed, 2 Feb 2011 00:29:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=python.org; s=200901; t=1296602952; bh=SjeURYFJ0JANVg28+rd+7SNHYu2uW4+khr521TTR24c=; h=MIME-Version:Date:Message-ID:Subject:From:To:Reply-To:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Content-Type:Content-Transfer-Encoding:Sender; b=u3Vd1rser4Yn6sV8mw8GsXc9yRHbfI8ct22qpxaml8r0lxcbKfbHnjjLVdST1kZY3 0MKS5mryz3ElA42r6P3S60hM+tWW0uz4xFk2AVUCrIeWt3+tpHWgYx3oXKHdTtdKRN ZgYWWK3rX3RoCiBOhAKDddtq25GWXs4jg2p75OVc= X-Original-To: python-announce-list at python.org Delivered-To: python-announce-list at mail.python.org Received: from albatross.python.org (localhost [127.0.0.1]) by mail.python.org (Postfix) with ESMTP id 51C0CEE9B8 for ; Tue, 1 Feb 2011 23:11:46 +0100 (CET) X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'bug': 0.02; 'subject:ANN': 0.02; 'subject:Python': 0.05; '2.x': 0.05; 'implements': 0.05; 'library': 0.06; '3.x': 0.07; 'derivatives': 0.07; 'tracker': 0.07; 'url:googlecode': 0.07; 'python': 0.08; 'url:pypi': 0.08; 'arithmetic': 0.09; 'url:code': 0.15; '0.17': 0.16; 'codebase,': 0.16; 'subject:0.17': 0.16; 'url:changes': 0.16; 'url:issues': 0.16; 'url:svn': 0.16; 'url:trunk': 0.16; 'downloaded': 0.17; 'url:doc': 0.19; '2.4': 0.23; 'version': 0.24; 'url:group': 0.25; 'function': 0.27; 'functions.': 0.28; 'project': 0.29; 'message- id:@mail.gmail.com': 0.29; 'url:)': 0.29; 'thanks': 0.30; 'dropped': 0.30; 'standalone': 0.30; 'all,': 0.30; 'contributed': 0.31; 'requires': 0.33; 'version.': 0.34; 'url:google': 0.34; '2.5': 0.36; 'mathematical': 0.36; 'to:addr:python-announce-list': 0.36; 'used': 0.36; 'set': 0.37; 'list:': 0.37; 'works': 0.38; 'received:209.85': 0.38; 'received:google.com': 0.38; 'zero': 0.38; 'url:org': 0.38; 'subject:: ': 0.39; 'url:python': 0.39; 'to:addr:python.org': 0.40; 'comments': 0.40; 'url:p': 0.60; 'to:2**2': 0.64; 'url:0': 0.64; 'details,': 0.65; 'website:': 0.71; 'to:no real name:2**2': 0.75; 'url:17': 0.76; 'news': 0.78; 'fredrik': 0.91 Received: from localhost (HELO mail.python.org) (127.0.0.1) by albatross.python.org with SMTP; 01 Feb 2011 23:11:46 +0100 Received: from mail-qw0-f46.google.com (mail-qw0-f46.google.com [209.85.216.46]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mail.python.org (Postfix) with ESMTPS for ; Tue, 1 Feb 2011 23:11:45 +0100 (CET) Received: by qwa26 with SMTP id 26so7212791qwa.19 for ; Tue, 01 Feb 2011 14:11:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:date:message-id:subject:from:to :content-type; bh=6yZ49a5xiXwaVsySNvnihUrvwjNRAycqDeQTj98mFN4=; b=TFkhQdC5AeC7EecRVy8bO+ZVjyDOniTJB9u5S4BVnuXSAIAa8U58xnIPqCNAEojT5M 2bA2NizGpC/zHNyQLa/yu92nhY85wb2Wv5FW72QkL9kklUpzkYFZp4/vi4nidCwFoA2R 7qa0fa3fksMnYLpIyfOE5Vv0NeyWrZKmxt/Jc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=qdEow0h50/OGUU89qp7muEE5FACELAhcuIpeNUeeV8srLOc4roz/kZHOS7NooIktaG 8lMo4dIEgm6aPAqeAxssYBAiQ5YDk/k0YxSsZovPJFV1incJEWpc9HIuuS9922zQUHeW J9J1GUwfUr/2QjPYed8ZCEQJkzSyAaSvGNFpI= MIME-Version: 1.0 Received: by 10.229.236.134 with SMTP id kk6mr7177714qcb.93.1296598304793; Tue, 01 Feb 2011 14:11:44 -0800 (PST) Received: by 10.229.245.209 with HTTP; Tue, 1 Feb 2011 14:11:44 -0800 (PST) Date: Tue, 1 Feb 2011 23:11:44 +0100 Message-ID: Subject: ANN: mpmath 0.17 (Python 3 support and more) From: Fredrik Johansson To: mpmath at googlegroups.com, sympy at googlegroups.com, sage-devel at googlegroups.com, python-announce-list at python.org X-Mailman-Approved-At: Wed, 02 Feb 2011 00:22:14 +0100 X-BeenThere: python-announce-list at python.org X-Mailman-Version: 2.1.12 Precedence: list Reply-To: python-list at python.org List-Id: Announcement-only list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: python-announce-list-bounces+lac=openend.se at python.org Errors-To: python-announce-list-bounces+lac=openend.se at python.org X-Greylist: Sender succeeded STARTTLS authentication, not delayed by milter-greylist-3.0 (theraft.openend.se [83.140.78.130]); Wed, 02 Feb 2011 00:29:14 +0100 (CET) X-DSPAM-Result: Innocent X-DSPAM-Processed: Wed Feb 2 00:29:19 2011 X-DSPAM-Confidence: 0.9994 X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 1004,4d48974f95865701517258 Hi all, Version 0.17 of mpmath is now available on the project website: http://code.google.com/p/mpmath/ It can also be downloaded from the Python Package Index: http://pypi.python.org/pypi/mpmath/0.17 Mpmath is a pure-Python library for arbitrary-precision floating-point arithmetic that implements an extensive set of mathematical functions. It can be used as a standalone library or via SymPy (http://code.google.com/p/sympy/), and is also available as a standard component of Sage (http://sagemath.org/). The major news in 0.17 is that mpmath now works with Python 3. To support both Python 2.x and 3.x with the same codebase, compatibility with Python 2.4 has been dropped (mpmath now requires 2.5 or higher). New functionality in mpmath 0.17 includes an implementation of the Lerch transcendent, Riemann zeta zero counting, and improved support for evaluating derivatives of the Hurwitz zeta function and related functions. Many thanks to Juan Arias de Reyna and Case Vanhorsen who contributed to this version. For more details, see the changelog: http://mpmath.googlecode.com/svn/tags/0.17/CHANGES Extensive documentation is available at: http://mpmath.googlecode.com/svn/trunk/doc/build/index.html or http://mpmath.googlecode.com/svn/tags/0.17/doc/build/index.html Bug reports and other comments are welcome on the issue tracker at http://code.google.com/p/mpmath/issues/list or the mpmath mailing list: http://groups.google.com/group/mpmath Enjoy, Fredrik Johansson - -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/ ------- End of Forwarded Message From tvoglou at gmail.com Wed Feb 2 00:56:26 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Wed, 2 Feb 2011 01:56:26 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: Hello Paolo, Dunno if I am in the correct direction but till now I found two things 1) test_genc was throwing an error on fixup_c_type (complaining about conflicting field type between HLOCAL and pointer to char array) so I changed it to an opaque pointer (and the HANDLE type) 2) test_genc passed the tests, so the next step was to change the env to use the 64bit libs changing from vsvars32.bat to vcvarsall.bat with amd64 param in translator/platform/windows.py # _get_msvc_env(vsver): After that change I had lot's of linking issues like [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp_Py_BuildValue referenced in function malloc_counters [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp_PyMethod_New referenced in function gencfunc_descr_get [platform:ERROR] testing_25.obj : error LNK2019: unresolved external symbol __imp__Py_NoneStruct referenced in function gencfunc_descr_get Any idea ? Thanks, Tasos On Wed, Feb 2, 2011 at 1:35 AM, Paolo Giarrusso wrote: > 2011/2/1 Ren? Dudfield : >> Hi, >> >> A naive guess would be the different sizes between 64bit windows, and >> 64 bit linux? ?(long int is 32bit on windows and 64bit on linux) >> >> From http://en.wikipedia.org/wiki/64-bit >> >> 64-bit data models >> Data model ? ? ?short (integer) int ? ? long (integer) ?long >> long ? ?pointers/size_t Sample operating systems >> LLP64/ >> IL32P64 16 ? ? ?32 ? ? ?32 ? ? ?64 ? ? ?64 ? ? ?Microsoft Windows (X64/IA-64) >> LP64/ >> I32LP64 16 ? ? ?32 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?Most Unix and Unix-like systems, e.g. Solaris, >> Linux, and Mac OS X >> ILP64 ? 16 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?HAL Computer Systems port of Solaris to SPARC64 >> SILP64 ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?64 ? ? ?Unicos > > From what Amaury said, you need to identify places where long is used > and it is assumed that it has 64bits. A first guess, in general, is > that _every_ use of long assumes it to be 64bit on 64bits systems. > > What about using intptr_t (or uintptr_t) from stdint.h? IOW, you'd > simply replace long by this type everywhere, including in functions > which generate C code through some library. I don't know the details, > could somebody help here? > IIRC this header was introduced in C99, and for sure this type is > optional - i.e., it must be declared if it is available at all. > > Of course, there might be systems where this type is not available, so > you need to provide a fallback (in a configure-like fashion). If this > header is available on (modern) Windows environments (the only system > where long is 32 bits), using "long" as fallback is safe. > > Cheers, > -- > Paolo Giarrusso - Ph.D. Student > http://www.informatik.uni-marburg.de/~pgiarrusso/ > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From amauryfa at gmail.com Wed Feb 2 01:06:56 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 2 Feb 2011 01:06:56 +0100 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: 2011/2/2 Tasos Vogiatzoglou : > After that change I had lot's of linking issues like > > [platform:ERROR] testing_25.obj : error LNK2019: unresolved external > symbol __imp_Py_BuildValue referenced in function malloc_counters > [platform:ERROR] testing_25.obj : error LNK2019: unresolved external > symbol __imp_PyMethod_New referenced in function gencfunc_descr_get > [platform:ERROR] testing_25.obj : error LNK2019: unresolved external > symbol __imp__Py_NoneStruct referenced in function gencfunc_descr_get > > Any idea ? You are certainly building an extension module for the host Python (the one which runs the tests). You should check that the link command finds the correct Python27.lib, corresponding to your installation of Python. btw, are you really running a 64bit Python? -- Amaury Forgeot d'Arc From amauryfa at gmail.com Wed Feb 2 01:02:22 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 2 Feb 2011 01:02:22 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: <201102012351.p11NpaJk011803@theraft.openend.se> References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: Hi, 2011/2/2 Laura Creighton : > running the test suite of this package might be useful. Some user already tried it one year ago, this even triggered two bugs in the early version of the JIT: http://codespeak.net/pipermail/pypy-dev/2010q1/005722.html I don't remember how fast is was, though. -- Amaury Forgeot d'Arc From tvoglou at gmail.com Wed Feb 2 01:15:10 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Wed, 2 Feb 2011 02:15:10 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: Amaury, You are right. My mistake. It was picking the 2.6 32bit python. Now is working and tests are passing Thanks, Tasos On Wed, Feb 2, 2011 at 2:06 AM, Amaury Forgeot d'Arc wrote: > 2011/2/2 Tasos Vogiatzoglou : >> After that change I had lot's of linking issues like >> >> [platform:ERROR] testing_25.obj : error LNK2019: unresolved external >> symbol __imp_Py_BuildValue referenced in function malloc_counters >> [platform:ERROR] testing_25.obj : error LNK2019: unresolved external >> symbol __imp_PyMethod_New referenced in function gencfunc_descr_get >> [platform:ERROR] testing_25.obj : error LNK2019: unresolved external >> symbol __imp__Py_NoneStruct referenced in function gencfunc_descr_get >> >> Any idea ? > > You are certainly building an extension module for the host Python > (the one which runs the tests). > You should check that the link command finds the correct Python27.lib, > corresponding to your > installation of Python. > btw, are you really running a 64bit Python? > > -- > Amaury Forgeot d'Arc > From orangewarrior at gmail.com Wed Feb 2 03:09:09 2011 From: orangewarrior at gmail.com (=?iso-8859-2?q?=A3ukasz_Ligowski?=) Date: Wed, 2 Feb 2011 03:09:09 +0100 Subject: [pypy-dev] pypy jit Message-ID: <201102020309.09640.orangewarrior@gmail.com> Hi, First of all I'd like to thank all developers for good work on PyPy ;) I'd like to ask are there any rules of thumb to write code that PyPy JIT can easily optimize? Or maybe which constructs are hardly optimizable by JIT so it's better to avoid them? Best regards, Lukasz Ligowski From benjamin at python.org Wed Feb 2 03:18:31 2011 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 1 Feb 2011 20:18:31 -0600 Subject: [pypy-dev] pypy jit In-Reply-To: <201102020309.09640.orangewarrior@gmail.com> References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: 2011/2/1 ?ukasz Ligowski : > Hi, > > First of all I'd like to thank all developers for good work on PyPy ;) > > I'd like to ask are there any rules of thumb to write code that PyPy JIT can > easily optimize? Or maybe which constructs are hardly optimizable by JIT so > it's better to avoid them? https://codespeak.net/viewvc/pypy/extradoc/talk/pycon2010/crossinterp/talk.pdf?view=log Mostly things you'd normally avoid anyway like sys._getframe(). -- Regards, Benjamin From jacob at openend.se Wed Feb 2 15:07:54 2011 From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=) Date: Wed, 2 Feb 2011 15:07:54 +0100 Subject: [pypy-dev] pypy jit In-Reply-To: <201102020309.09640.orangewarrior@gmail.com> References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: <201102021507.55292.jacob@openend.se> onsdag 02 februari 2011 03.09.09 skrev ?ukasz Ligowski: > > I'd like to ask are there any rules of thumb to write code that PyPy JIT > can easily optimize? Or maybe which constructs are hardly optimizable by > JIT so it's better to avoid them? If you want absolutely best performance, write your code in a simple, straight forward way, trying to keep your most used loops free of branches. if a: for x in range(big number): calculate something else: for x in range(big number): calculate something else generates better code than for x in range(big number): if a: calculate something else: caculate something else In the second case, there will be a guard inside the loop that has to be evaluated every time through. Jacob Hall?n From pgiarrusso at mathematik.uni-marburg.de Wed Feb 2 15:08:07 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Wed, 2 Feb 2011 15:08:07 +0100 Subject: [pypy-dev] pypy jit In-Reply-To: <201102021507.55292.jacob@openend.se> References: <201102020309.09640.orangewarrior@gmail.com> <201102021507.55292.jacob@openend.se> Message-ID: On Wed, Feb 2, 2011 at 15:07, Jacob Hall?n wrote: > onsdag 02 februari 2011 03.09.09 skrev ??ukasz Ligowski: >> >> I'd like to ask are there any rules of thumb to write code that PyPy JIT >> can easily optimize? Or maybe which constructs are hardly optimizable by >> JIT so it's better to avoid them? > > If you want absolutely best performance, write your code in a simple, straight > forward way, trying to keep your most used loops free of branches. > > if a: > ? ?for x in range(big number): > ? ? ? ?calculate something > else: > ? ?for x in range(big number): > ? ? ? ?calculate something else > > generates better code than > > for x in range(big number): > ? ?if a: > ? ? ? ?calculate something > ? ?else: > ? ? ? ?caculate something else > > In the second case, there will be a guard inside the loop that has to be > evaluated every time through. Is there a plan to automate this optimization, or are there problems to extend it to tracing JIT compilation? I think the standard name is code hoisting and other compilers do it. -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From alex.gaynor at gmail.com Wed Feb 2 15:06:32 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 2 Feb 2011 09:06:32 -0500 Subject: [pypy-dev] pypy jit In-Reply-To: <201102021507.55292.jacob@openend.se> References: <201102020309.09640.orangewarrior@gmail.com> <201102021507.55292.jacob@openend.se> Message-ID: On Wed, Feb 2, 2011 at 9:07 AM, Jacob Hall?n wrote: > onsdag 02 februari 2011 03.09.09 skrev ?ukasz Ligowski: > > > > I'd like to ask are there any rules of thumb to write code that PyPy JIT > > can easily optimize? Or maybe which constructs are hardly optimizable by > > JIT so it's better to avoid them? > > If you want absolutely best performance, write your code in a simple, > straight > forward way, trying to keep your most used loops free of branches. > > if a: > for x in range(big number): > calculate something > else: > for x in range(big number): > calculate something else > > generates better code than > > for x in range(big number): > if a: > calculate something > else: > caculate something else > > In the second case, there will be a guard inside the loop that has to be > evaluated every time through. > > Jacob Hall?n > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > If you happen to be doing numerical calculations using the array module instead of lists will help. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Wed Feb 2 16:47:53 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 2 Feb 2011 17:47:53 +0200 Subject: [pypy-dev] pypy jit In-Reply-To: <201102021507.55292.jacob@openend.se> References: <201102020309.09640.orangewarrior@gmail.com> <201102021507.55292.jacob@openend.se> Message-ID: On Wed, Feb 2, 2011 at 4:07 PM, Jacob Hall?n wrote: > onsdag 02 februari 2011 03.09.09 skrev ??ukasz Ligowski: >> >> I'd like to ask are there any rules of thumb to write code that PyPy JIT >> can easily optimize? Or maybe which constructs are hardly optimizable by >> JIT so it's better to avoid them? > > If you want absolutely best performance, write your code in a simple, straight > forward way, trying to keep your most used loops free of branches. > > if a: > ? ?for x in range(big number): > ? ? ? ?calculate something > else: > ? ?for x in range(big number): > ? ? ? ?calculate something else > > generates better code than > > for x in range(big number): > ? ?if a: > ? ? ? ?calculate something > ? ?else: > ? ? ? ?caculate something else > > In the second case, there will be a guard inside the loop that has to be > evaluated every time through. You have outdated information Jacob, not any more :) > > Jacob Hall?n > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From fijall at gmail.com Wed Feb 2 16:48:50 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 2 Feb 2011 17:48:50 +0200 Subject: [pypy-dev] pypy jit In-Reply-To: <201102020309.09640.orangewarrior@gmail.com> References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: 2011/2/2 ?ukasz Ligowski : > Hi, > > First of all I'd like to thank all developers for good work on PyPy ;) > > I'd like to ask are there any rules of thumb to write code that PyPy JIT can > easily optimize? Or maybe which constructs are hardly optimizable by JIT so > it's better to avoid them? I should probably right it down somewhere. As of now there is no such guide. However, the rule of thumb is simple is better than complex. > > Best regards, > Lukasz Ligowski > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From fredrik.johansson at gmail.com Thu Feb 3 00:02:15 2011 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Thu, 3 Feb 2011 00:02:15 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: Hello, On Wed, Feb 2, 2011 at 1:02 AM, Amaury Forgeot d'Arc wrote: > Hi, > > 2011/2/2 Laura Creighton : >> running the test suite of this package might be useful. > > Some user already tried it one year ago, this even triggered two bugs > in the early version of the JIT: > http://codespeak.net/pipermail/pypy-dev/2010q1/005722.html Indeed (that user was me, and it was nice to see the problem fixed so quickly!). There is one incompatibility left between CPython and PyPy that breaks some functions in mpmath. In CPython, round() and functions in the math module accept (and convert) inputs of any custom type with a __float__ method, whereas they trigger a TypeError in PyPy (1.4). > I don't remember how fast is was, though. On win32, PyPy 1.4 is about 40% slower than CPython 2.6 for running the entire set of tests. This is probably mostly due to the slowness of longs in PyPy. However, much of the regular floating point code seems to be slower in PyPy as well. Fredrik From amauryfa at gmail.com Thu Feb 3 00:10:46 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 3 Feb 2011 00:10:46 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: 2011/2/3 Fredrik Johansson : > Indeed (that user was me, and it was nice to see the problem fixed so > quickly!). There is one incompatibility left between CPython and PyPy > that breaks some functions in mpmath. In CPython, round() and > functions in the math module accept (and convert) inputs of any custom > type with a __float__ method, whereas they trigger a TypeError in PyPy > (1.4). It seems corrected today; the script below indeed fails with pypy-1.4. >>>> class C: .... def __float__(self): return 42.0 .... >>>> x = C() >>>> import math >>>> math.sqrt(x) 6.48074069840786 >> I don't remember how fast is was, though. > > On win32, PyPy 1.4 is about 40% slower than CPython 2.6 for running > the entire set of tests. This is probably mostly due to the slowness > of longs in PyPy. However, much of the regular floating point code > seems to be slower in PyPy as well. The standard operations, or the math module? It seems that every function in the math module releases the GIL. This may explain some slowness. -- Amaury Forgeot d'Arc From pgiarrusso at mathematik.uni-marburg.de Thu Feb 3 00:28:41 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Thu, 3 Feb 2011 00:28:41 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 00:10, Amaury Forgeot d'Arc wrote: > 2011/2/3 Fredrik Johansson : >> Indeed (that user was me, and it was nice to see the problem fixed so >> quickly!). There is one incompatibility left between CPython and PyPy >> that breaks some functions in mpmath. In CPython, round() and >> functions in the math module accept (and convert) inputs of any custom >> type with a __float__ method, whereas they trigger a TypeError in PyPy >> (1.4). > > It seems corrected today; the script below indeed fails with pypy-1.4. According to Fredik, the script is supposed to pass. By "corrected" you mean it's fixed on the trunk? >>>>> class C: > .... ? def __float__(self): return 42.0 > .... >>>>> x = C() >>>>> import math >>>>> math.sqrt(x) > 6.48074069840786 >>> I don't remember how fast is was, though. >> >> On win32, PyPy 1.4 is about 40% slower than CPython 2.6 for running >> the entire set of tests. This is probably mostly due to the slowness >> of longs in PyPy. However, much of the regular floating point code >> seems to be slower in PyPy as well. > > The standard operations, or the math module? > It seems that every function in the math module releases the GIL. Does that module live in pypy/module/math/? Reading the source doesn't show obvious interactions with the GIL, so I guess that for many other modules the GIL is released automatically, potentially even for other fast functions. > This may explain some slowness. In other words, a simple call to sin()/cos()/ceil() triggers a cache flush (costing around ~100 cycles) because of releasing a lock, if not a system call (costing thousands of cycles), say if the lock was contended? If those functions were lifted to operate on lists they should release the lock for long lists, otherwise it seems a bad idea. In other words, I'd propose to remove the unlocking/relocking if possible. Can that affect correctness for some weird reason? I'd guess not, but I'm not familiar with the code enough to get fully what happens. Best regards -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From fredrik.johansson at gmail.com Thu Feb 3 00:29:30 2011 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Thu, 3 Feb 2011 00:29:30 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 12:10 AM, Amaury Forgeot d'Arc wrote: > 2011/2/3 Fredrik Johansson : >> Indeed (that user was me, and it was nice to see the problem fixed so >> quickly!). There is one incompatibility left between CPython and PyPy >> that breaks some functions in mpmath. In CPython, round() and >> functions in the math module accept (and convert) inputs of any custom >> type with a __float__ method, whereas they trigger a TypeError in PyPy >> (1.4). > > It seems corrected today; the script below indeed fails with pypy-1.4. Ah, that's nice. Does round() work as well? >> On win32, PyPy 1.4 is about 40% slower than CPython 2.6 for running >> the entire set of tests. This is probably mostly due to the slowness >> of longs in PyPy. However, much of the regular floating point code >> seems to be slower in PyPy as well. > > The standard operations, or the math module? > It seems that every function in the math module releases the GIL. > This may explain some slowness. I have only looked at some quick macro level timings. Generally a lot of overhead for function calls and type checking is involved (although I would hope that PyPy can kill precisely some of that eventually). Fredrik From alex.gaynor at gmail.com Thu Feb 3 00:32:12 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Wed, 2 Feb 2011 18:32:12 -0500 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Wed, Feb 2, 2011 at 6:29 PM, Fredrik Johansson < fredrik.johansson at gmail.com> wrote: > On Thu, Feb 3, 2011 at 12:10 AM, Amaury Forgeot d'Arc > wrote: > > 2011/2/3 Fredrik Johansson : > >> Indeed (that user was me, and it was nice to see the problem fixed so > >> quickly!). There is one incompatibility left between CPython and PyPy > >> that breaks some functions in mpmath. In CPython, round() and > >> functions in the math module accept (and convert) inputs of any custom > >> type with a __float__ method, whereas they trigger a TypeError in PyPy > >> (1.4). > > > > It seems corrected today; the script below indeed fails with pypy-1.4. > > Ah, that's nice. Does round() work as well? > > >> On win32, PyPy 1.4 is about 40% slower than CPython 2.6 for running > >> the entire set of tests. This is probably mostly due to the slowness > >> of longs in PyPy. However, much of the regular floating point code > >> seems to be slower in PyPy as well. > > > > The standard operations, or the math module? > > It seems that every function in the math module releases the GIL. > > This may explain some slowness. > > I have only looked at some quick macro level timings. Generally a lot > of overhead for function calls and type checking is involved (although > I would hope that PyPy can kill precisely some of that eventually). > > Fredrik > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > If you have any benchmarks where we are slower that don't involve longs that'd be great, since for float operations we really should be able to beat up on CPython. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Thu Feb 3 00:42:09 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 3 Feb 2011 00:42:09 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: 2011/2/3 Paolo Giarrusso : >> It seems that every function in the math module releases the GIL. > Does that module live in pypy/module/math/? Reading the source doesn't > show obvious interactions with the GIL, so I guess that for many other > modules the GIL is released automatically, potentially even for other > fast functions. Fortunately, I was wrong, sorry. The functions of the math module don't release the GIL. The low-level operations are in pypy/rpython/lltypesystem/module/ll_math.py, all functions are marked with "sandboxsafe=True" which, according to a comment in rffi.py, ensures that the GIL won't be released. -- Amaury Forgeot d'Arc From fredrik.johansson at gmail.com Thu Feb 3 00:43:57 2011 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Thu, 3 Feb 2011 00:43:57 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor wrote: > If you have any benchmarks where we are slower that don't involve longs > that'd be great, since for float operations we really should be able to beat > up on CPython. A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(1000000)) PyPy 1.4 runs this about 2.4 times slower than CPython. Fredrik From fijall at gmail.com Thu Feb 3 08:01:08 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 3 Feb 2011 09:01:08 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson wrote: > On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor wrote: >> If you have any benchmarks where we are slower that don't involve longs >> that'd be great, since for float operations we really should be able to beat >> up on CPython. > > A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(1000000)) A brief follow up. * PyPy trunk is faster (by quite a bit). * I noticed that you happily use mixture of old and new style classes. As of now this is a really bad case for PyPy. Example: >>>> [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] [True, True, True, True, False, False, True, True, False, True, True, True, True, True, True] when I replace it with newstyle classes it runs much faster Other things that speed up both CPython and PyPy: * Put this things into function instead of at global scope * Use list comprehension instead of generator expression. That all should make PyPy 3x faster than CPython. > > PyPy 1.4 runs this about 2.4 times slower than CPython. > > Fredrik > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From fredrik.johansson at gmail.com Thu Feb 3 10:56:13 2011 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Thu, 3 Feb 2011 10:56:13 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: > On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson > wrote: >> On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor wrote: >>> If you have any benchmarks where we are slower that don't involve longs >>> that'd be great, since for float operations we really should be able to beat >>> up on CPython. >> >> A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(1000000)) > > A brief follow up. > > * PyPy trunk is faster (by quite a bit). > > * I noticed that you happily use mixture of old and new style classes. > As of now this is a really bad case for PyPy. Example: > >>>>> [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] > [True, True, True, True, False, False, True, True, False, True, True, > True, True, True, True] > > when I replace it with newstyle classes it runs much faster Interesting. The mixture of old and new style classes is a mistake, of course. I'm going to add a test to make sure this doesn't happen. Thanks for pointing it out. In fact this speeds up another benchmark I did -- [fp.lambertw(k) for k in xrange(50000)]-- by 10x, which is quite a ridiculous ratio! > Other things that speed up both CPython and PyPy: > > * Put this things into function instead of at global scope Do you mean in the benchmark or did you have some other code you saw in mind? > * Use list comprehension instead of generator expression. I hope PyPy can do more in the future to speed up generator expressions. Fredrik From fijall at gmail.com Thu Feb 3 11:14:53 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 3 Feb 2011 12:14:53 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: > On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: >> On Thu, Feb 3, 2011 at 1:43 AM, Fredrik Johansson >> wrote: >>> On Thu, Feb 3, 2011 at 12:32 AM, Alex Gaynor wrote: >>>> If you have any benchmarks where we are slower that don't involve longs >>>> that'd be great, since for float operations we really should be able to beat >>>> up on CPython. >>> >>> A simple example: mpmath.fp.fsum(mpmath.fp.sqrt(k) for k in xrange(1000000)) >> >> A brief follow up. >> >> * PyPy trunk is faster (by quite a bit). >> >> * I noticed that you happily use mixture of old and new style classes. >> As of now this is a really bad case for PyPy. Example: >> >>>>>> [isinstance(i, type) for i in mpmath.fp.__class__.__mro__] >> [True, True, True, True, False, False, True, True, False, True, True, >> True, True, True, True] >> >> when I replace it with newstyle classes it runs much faster > > Interesting. The mixture of old and new style classes is a mistake, of > course. I'm going to add a test to make sure this doesn't happen. > Thanks for pointing it out. > > In fact this speeds up another benchmark I did -- [fp.lambertw(k) for > k in xrange(50000)]-- by 10x, which is quite a ridiculous ratio! Mixture of old and new style classes is not only preventing us from doing optimizations but also hits a bad case of tradeoffs. However, we decided we don't care that much. You should use new style classes anyway :) > >> Other things that speed up both CPython and PyPy: >> >> * Put this things into function instead of at global scope > > Do you mean in the benchmark or did you have some other code you saw in mind? The benchmark. > >> * Use list comprehension instead of generator expression. > > I hope PyPy can do more in the future to speed up generator expressions. It doesn't speed up things by much. Yeah, I can imagine we can improve on this, but it's also a bit hard. > > Fredrik > From fredrik.johansson at gmail.com Thu Feb 3 11:23:02 2011 From: fredrik.johansson at gmail.com (Fredrik Johansson) Date: Thu, 3 Feb 2011 11:23:02 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 11:14 AM, Maciej Fijalkowski wrote: > Mixture of old and new style classes is not only preventing us from > doing optimizations but also hits a bad case of tradeoffs. However, we > decided we don't care that much. You should use new style classes > anyway :) Yes, this is perfectly reasonable. You should make PyPy print warnings when it encounters mixed-type classes :) Fredrik From fijall at gmail.com Thu Feb 3 11:26:46 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 3 Feb 2011 12:26:46 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 12:23 PM, Fredrik Johansson wrote: > On Thu, Feb 3, 2011 at 11:14 AM, Maciej Fijalkowski wrote: >> Mixture of old and new style classes is not only preventing us from >> doing optimizations but also hits a bad case of tradeoffs. However, we >> decided we don't care that much. You should use new style classes >> anyway :) > > Yes, this is perfectly reasonable. You should make PyPy print warnings > when it encounters mixed-type classes :) > > Fredrik > That's not a bad idea actually. Maybe with something like some -X option? From amauryfa at gmail.com Thu Feb 3 13:10:43 2011 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 3 Feb 2011 13:10:43 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: 2011/2/3 Maciej Fijalkowski : > You should make PyPy print warnings >> when it encounters mixed-type classes :) >> > That's not a bad idea actually. Maybe with something like some -X option? If we use the warnings module to emit JitWarnings, the option could be -Wd::JitWarning -- Amaury Forgeot d'Arc From fijall at gmail.com Thu Feb 3 13:13:55 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 3 Feb 2011 14:13:55 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 2:10 PM, Amaury Forgeot d'Arc wrote: > 2011/2/3 Maciej Fijalkowski : >> You should make PyPy print warnings >>> when it encounters mixed-type classes :) >>> >> That's not a bad idea actually. Maybe with something like some -X option? > > If we use the warnings module to emit JitWarnings, the option could be > -Wd::JitWarning I'm not sure about warnings module. How about --jit warnings=1 ? That would fit with other jit options. I was more thinking about this being optional > > -- > Amaury Forgeot d'Arc > From anto.cuni at gmail.com Thu Feb 3 13:21:57 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 03 Feb 2011 13:21:57 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: <4D4A9DE5.1070601@gmail.com> On 03/02/11 13:13, Maciej Fijalkowski wrote: > I'm not sure about warnings module. How about > --jit warnings=1 > ? > > That would fit with other jit options. not really. The other jit options really belongs to the JIT engine, while this one is dependent on the Python interpreter ciao, Anto From fijall at gmail.com Thu Feb 3 13:22:57 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 3 Feb 2011 14:22:57 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: <4D4A9DE5.1070601@gmail.com> References: <201102012351.p11NpaJk011803@theraft.openend.se> <4D4A9DE5.1070601@gmail.com> Message-ID: On Thu, Feb 3, 2011 at 2:21 PM, Antonio Cuni wrote: > On 03/02/11 13:13, Maciej Fijalkowski wrote: > >> I'm not sure about warnings module. How about >> --jit warnings=1 >> ? >> >> That would fit with other jit options. > > not really. The other jit options really belongs to the JIT engine, while this > one is dependent on the Python interpreter true. > > ciao, > Anto > From arigo at tunes.org Thu Feb 3 13:30:16 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 3 Feb 2011 13:30:16 +0100 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: Hi Tasos, On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou wrote: > You are right. My mistake. It was picking the 2.6 32bit python. Now is > working and tests are passing A few extra notes. What is needed to make this work is three-fold: * make sure that in the C code, "intptr_t" is generated instead of "long" for lltype.Signed, and similarly "uintptr_t" instead of "unsigned long" for lltype.Unsigned. This involves mainly fixing translator/c/primitive.py to declare types using "intptr_t" instead of "long", and probably fixing a number of other uses of "long" in the .py files in that directory. * make sure that in the hand-written headers translator/c/src/*.h, we use "intptr_t" instead of "long" whenever it's needed (mostly everywhere, I suppose, but not absolutely everywhere, e.g. not in calls to external C functions that are declared to take or return a long). The changes above should fix the generation of code. Now the third part is the longest, probably. You need to distinguish the two types used in RPython, which are lltype.Signed and rffi.LONG, and which are equal so far. Of course rffi.LONG should remain equivalent to the C long. This probably needs looking around in all modules and in pypy/rlib/, to make sure that we use the correct one, either lltype.Signed or rffi.LONG. Fortunately, errors resulting from this confusion will probably just give translation-time errors, so we can fix the places one after the other. But you need to carefully check the declaration of external functions that are implemented in translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this one happens to use mostly rffi.INT, so it should work right now, with the exception of LONG_STRUCT that uses a lltype.Signed). A bient?t, Armin. From tvoglou at gmail.com Thu Feb 3 15:09:00 2011 From: tvoglou at gmail.com (Tasos Vogiatzoglou) Date: Thu, 3 Feb 2011 16:09:00 +0200 Subject: [pypy-dev] Link errors while translating with VS2010 and 64bit In-Reply-To: References: <-3229585076398878208@unknownmsgid> Message-ID: <-4189686393236333789@unknownmsgid> Armin, thanks for the directions. I'll start working on them and I'll report back. Regards, Tasos On 3 ??? 2011, at 14:30, Armin Rigo wrote: > Hi Tasos, > > On Wed, Feb 2, 2011 at 1:15 AM, Tasos Vogiatzoglou wrote: >> You are right. My mistake. It was picking the 2.6 32bit python. Now is >> working and tests are passing > > A few extra notes. What is needed to make this work is three-fold: > > * make sure that in the C code, "intptr_t" is generated instead of > "long" for lltype.Signed, and similarly "uintptr_t" instead of > "unsigned long" for lltype.Unsigned. This involves mainly fixing > translator/c/primitive.py to declare types using "intptr_t" instead of > "long", and probably fixing a number of other uses of "long" in the > .py files in that directory. > > * make sure that in the hand-written headers translator/c/src/*.h, we > use "intptr_t" instead of "long" whenever it's needed (mostly > everywhere, I suppose, but not absolutely everywhere, e.g. not in > calls to external C functions that are declared to take or return a > long). > > The changes above should fix the generation of code. Now the third > part is the longest, probably. You need to distinguish the two types > used in RPython, which are lltype.Signed and rffi.LONG, and which are > equal so far. Of course rffi.LONG should remain equivalent to the C > long. This probably needs looking around in all modules and in > pypy/rlib/, to make sure that we use the correct one, either > lltype.Signed or rffi.LONG. Fortunately, errors resulting from this > confusion will probably just give translation-time errors, so we can > fix the places one after the other. But you need to carefully check > the declaration of external functions that are implemented in > translator/c/src/*.h, like pypy/module/signal/interp_signal.py (this > one happens to use mostly rffi.INT, so it should work right now, with > the exception of LONG_STRUCT that uses a lltype.Signed). > > > A bient?t, > > Armin. From arigo at tunes.org Thu Feb 3 16:03:13 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 3 Feb 2011 16:03:13 +0100 Subject: [pypy-dev] pypy jit In-Reply-To: References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: Hi, On Wed, Feb 2, 2011 at 4:48 PM, Maciej Fijalkowski wrote: > As of now there is no such guide. However, the rule of thumb is simple > is better than complex. To some extend that is true; however, note that the second rule is "but even on messily complicated code the JIT can do something" :-) For example, "translate.py" was not written with the JIT in mind at all, but turns out to be twice as fast on PyPy. To get a better summary, I think that we can say that, to some extent, there is little point in spending ages tweaking your Python code to make it more JIT-friendly. If there is any Python code that would get seriously faster by some trivial tweaking, then it's somehow a bug, and we need to fix it. Of course, if you use introspection of the stack frames or even pdb (the Python debugger) all over the place, you need to expect the code to be hard to optimize for us. But I guess that it should somehow be clear. For the rest, any decision that can be resolved "locally" can nicely be optimized by the JIT, whatever the answer is (for example, "is this '+' going to just add integers, or does it invoke some __add__() method?"). A bient?t, Armin. From stefan_ml at behnel.de Thu Feb 3 17:15:53 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 03 Feb 2011 17:15:53 +0100 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: Maciej Fijalkowski, 03.02.2011 11:14: > On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: >> On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: >>> * Use list comprehension instead of generator expression. >> >> I hope PyPy can do more in the future to speed up generator expressions. > > It doesn't speed up things by much. Yeah, I can imagine we can improve > on this, but it's also a bit hard. Does PyPy generate inlined looping code for them if applicable? (e.g. for any(), all(), sum(), sorted(), ...) Stefan From alex.gaynor at gmail.com Thu Feb 3 17:25:41 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Thu, 3 Feb 2011 11:25:41 -0500 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 11:15 AM, Stefan Behnel wrote: > Maciej Fijalkowski, 03.02.2011 11:14: > > On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: > >> On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: > >>> * Use list comprehension instead of generator expression. > >> > >> I hope PyPy can do more in the future to speed up generator expressions. > > > > It doesn't speed up things by much. Yeah, I can imagine we can improve > > on this, but it's also a bit hard. > > Does PyPy generate inlined looping code for them if applicable? > > (e.g. for any(), all(), sum(), sorted(), ...) > > Stefan > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > No, it does not (with the exception of map() which has its own JIT, but that code is still not inlined into the caller). Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Fri Feb 4 07:41:07 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 4 Feb 2011 08:41:07 +0200 Subject: [pypy-dev] testing floating point In-Reply-To: References: <201102012351.p11NpaJk011803@theraft.openend.se> Message-ID: On Thu, Feb 3, 2011 at 6:15 PM, Stefan Behnel wrote: > Maciej Fijalkowski, 03.02.2011 11:14: >> On Thu, Feb 3, 2011 at 11:56 AM, Fredrik Johansson wrote: >>> On Thu, Feb 3, 2011 at 8:01 AM, Maciej Fijalkowski wrote: >>>> * Use list comprehension instead of generator expression. >>> >>> I hope PyPy can do more in the future to speed up generator expressions. >> >> It doesn't speed up things by much. Yeah, I can imagine we can improve >> on this, but it's also a bit hard. > > Does PyPy generate inlined looping code for them if applicable? > > (e.g. for any(), all(), sum(), sorted(), ...) We don't inline generator expr. The reason is that it's hard to judge when generator frame escapes. We can probably do something with it, but it's not done yet. > > Stefan > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From fijall at gmail.com Fri Feb 4 07:44:07 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 4 Feb 2011 08:44:07 +0200 Subject: [pypy-dev] pypy jit In-Reply-To: References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: On Thu, Feb 3, 2011 at 5:03 PM, Armin Rigo wrote: > Hi, > > On Wed, Feb 2, 2011 at 4:48 PM, Maciej Fijalkowski wrote: >> As of now there is no such guide. However, the rule of thumb is simple >> is better than complex. > > To some extend that is true; however, note that the second rule is > "but even on messily complicated code the JIT can do something" :-) > For example, "translate.py" was not written with the JIT in mind at > all, but turns out to be twice as fast on PyPy. > > To get a better summary, I think that we can say that, to some extent, > there is little point in spending ages tweaking your Python code to > make it more JIT-friendly. ?If there is any Python code that would get > seriously faster by some trivial tweaking, then it's somehow a bug, > and we need to fix it. ?Of course, if you use introspection of the > stack frames or even pdb (the Python debugger) all over the place, you > need to expect the code to be hard to optimize for us. ?But I guess > that it should somehow be clear. ?For the rest, any decision that can > be resolved "locally" can nicely be optimized by the JIT, whatever the > answer is (for example, "is this '+' going to just add integers, or > does it invoke some __add__() method?"). > I disagree a bit. The example would be newstylization of classes. This is making your code more JIT friendly and it's not trivial to optimize this from the JIT standpoint. > > A bient?t, > > Armin. > From arigo at tunes.org Fri Feb 4 17:12:55 2011 From: arigo at tunes.org (Armin Rigo) Date: Fri, 4 Feb 2011 17:12:55 +0100 Subject: [pypy-dev] pypy jit In-Reply-To: References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: Hi, On Fri, Feb 4, 2011 at 7:44 AM, Maciej Fijalkowski wrote: > I disagree a bit. The example would be newstylization of classes. This > is making your code more JIT friendly and it's not trivial to optimize > this from the JIT standpoint. "We know there is an issue there but we don't want to invest the large efforts to improve it because it's really the Python program's fault for using a mixture of deprecated and modern features"... Sure, you have a point. I was taking a rather theoretical point of view in my previous e-mail. A bient?t, Armin. From fijall at gmail.com Fri Feb 4 18:55:40 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Fri, 4 Feb 2011 19:55:40 +0200 Subject: [pypy-dev] pypy jit In-Reply-To: References: <201102020309.09640.orangewarrior@gmail.com> Message-ID: On Fri, Feb 4, 2011 at 6:12 PM, Armin Rigo wrote: > Hi, > > On Fri, Feb 4, 2011 at 7:44 AM, Maciej Fijalkowski wrote: >> I disagree a bit. The example would be newstylization of classes. This >> is making your code more JIT friendly and it's not trivial to optimize >> this from the JIT standpoint. > > "We know there is an issue there but we don't want to invest the large > efforts to improve it because it's really the Python program's fault > for using a mixture of deprecated and modern features"... ?Sure, you > have a point. ?I was taking a rather theoretical point of view in my > previous e-mail. > Thank you for summarizing :-) I started a wiki btw: https://bitbucket.org/pypy/pypy/wiki/JitFriendliness > > A bient?t, > > Armin. > From tismer at stackless.com Sat Feb 5 23:09:19 2011 From: tismer at stackless.com (Christian Tismer) Date: Sat, 05 Feb 2011 23:09:19 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ Message-ID: <4D4DCA8F.7070508@stackless.com> Howdy, studying the differences of PyPy vs. CPython, most seem to be fine; one thing where I an unsure is the __del__ behavior. I am not addressing its delayed call or the number it is called, this is similar to Jython and IronPython. But assigning to __del__ after a class is created, is that so hard to implement? It is clear to me that the existence of a __del__ heavily influences the whole code generation of a class. So as a naive thought, can we just abandon the whole compilation and start over (maybe even less)? The problem that I see is re-generation of already created instances, triggering some re-compilation. I would think of all compiled code always going through some "fuse code" that would trigger re-compilation if an assignment to __del__ happens. For me, this looks like a guard, that exists in other places of the JIT, where existing assumptions must check if they are still true, and otherwise start over. I have not read the current implementation, yet, because my reading is still quite impaired. But from the top of my head: Can it be that such a re-compile as I mentioned is not yet considered because that is not involving the JIT, but an already fixed part of the static compilation chain? I think not, since we are not in RPython, but at the translation of an applevel class, which is completely run-time compiled. So, where is my lack of seeing the problem? Or is it simply a lot of effort? If it is an efficiency concern, then my POV is: Better to be slow in a few cases of re-compilation, than to have to hunt for such delicate existing code. I would instead issue an optional warning to dynamic __del__ assignment instead, in order to let people avoid it in the future. regards, and thanks for correcting me - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From benjamin at python.org Sat Feb 5 23:47:50 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 5 Feb 2011 16:47:50 -0600 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <4D4DCA8F.7070508@stackless.com> References: <4D4DCA8F.7070508@stackless.com> Message-ID: 2011/2/5 Christian Tismer : > Howdy, > > studying the differences of PyPy vs. CPython, most seem to be fine; > one thing where I an unsure is the __del__ behavior. > > I am not addressing its delayed call or the number it is called, this > is similar to Jython and IronPython. > > But assigning to __del__ after a class is created, is that so hard > to implement? It's not a JIT problem rather a RPython/gc one. All the RPython classes with finalizers must be known at translation time. __del__ is expensive in the for gc. To implement user level __del__, a different underlying interp class is used which has its own __del__ which the gc will call. -- Regards, Benjamin From tismer at stackless.com Sun Feb 6 00:10:48 2011 From: tismer at stackless.com (Christian Tismer) Date: Sun, 06 Feb 2011 00:10:48 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: References: <4D4DCA8F.7070508@stackless.com> Message-ID: <4D4DD8F8.7080308@stackless.com> On 2/5/11 11:47 PM, Benjamin Peterson wrote: > 2011/2/5 Christian Tismer: >> Howdy, >> >> studying the differences of PyPy vs. CPython, most seem to be fine; >> one thing where I an unsure is the __del__ behavior. >> >> I am not addressing its delayed call or the number it is called, this >> is similar to Jython and IronPython. >> >> But assigning to __del__ after a class is created, is that so hard >> to implement? > It's not a JIT problem rather a RPython/gc one. All the RPython > classes with finalizers must be known at translation time. __del__ is > expensive in the for gc. To implement user level __del__, a different > underlying interp class is used which has its own __del__ which the gc > will call. I understand. Then having a __del__ is always expensive, I guess? I mean, does it involve overhead to have a __del__ at all, runtime or compile time? How feasible would it be to generate always two versions of the RPython class with a stub __del__ method, which calls a yet non-existing function? sorry if that is non-sense, but maybe something can be done to isolate these bad spots, without simply silently not calling them. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From benjamin at python.org Sun Feb 6 00:25:26 2011 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 5 Feb 2011 17:25:26 -0600 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <4D4DD8F8.7080308@stackless.com> References: <4D4DCA8F.7070508@stackless.com> <4D4DD8F8.7080308@stackless.com> Message-ID: 2011/2/5 Christian Tismer : > On 2/5/11 11:47 PM, Benjamin Peterson wrote: >> >> 2011/2/5 Christian Tismer: >>> >>> Howdy, >>> >>> studying the differences of PyPy vs. CPython, most seem to be fine; >>> one thing where I an unsure is the __del__ behavior. >>> >>> I am not addressing its delayed call or the number it is called, this >>> is similar to Jython and IronPython. >>> >>> But assigning to __del__ after a class is created, is that so hard >>> to implement? >> >> It's not a JIT problem rather a RPython/gc one. All the RPython >> classes with finalizers must be known at translation time. __del__ is >> expensive in the for gc. To implement user level __del__, a different >> underlying interp class is used which has its own __del__ which the gc >> will call. > > I understand. Then having a __del__ is always expensive, I guess? > I mean, does it involve overhead to have a __del__ at all, runtime > or compile time? It just means there's a lot of runtime bookkeeping in the GC for objects with __del__. > > How feasible would it be to generate always two versions of > the RPython class with a stub __del__ method, which calls a > yet non-existing function? > > sorry if that is non-sense, but maybe something can be done > to isolate these bad spots, without simply silently not calling them. Well, you do at least get a nice warning. I don't think this is too hard to work around in apps. There's also weakrefs. > > cheers - chris -- Regards, Benjamin From tismer at stackless.com Sun Feb 6 00:49:00 2011 From: tismer at stackless.com (Christian Tismer) Date: Sun, 06 Feb 2011 00:49:00 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: References: <4D4DCA8F.7070508@stackless.com> <4D4DD8F8.7080308@stackless.com> Message-ID: <4D4DE1EC.6020107@stackless.com> On 2/6/11 12:25 AM, Benjamin Peterson wrote: > 2011/2/5 Christian Tismer: >> On 2/5/11 11:47 PM, Benjamin Peterson wrote: >>> 2011/2/5 Christian Tismer: >>>> Howdy, >>>> >>>> studying the differences of PyPy vs. CPython, most seem to be fine; >>>> one thing where I an unsure is the __del__ behavior. >>>> >>>> I am not addressing its delayed call or the number it is called, this >>>> is similar to Jython and IronPython. >>>> >>>> But assigning to __del__ after a class is created, is that so hard >>>> to implement? >>> It's not a JIT problem rather a RPython/gc one. All the RPython >>> classes with finalizers must be known at translation time. __del__ is >>> expensive in the for gc. To implement user level __del__, a different >>> underlying interp class is used which has its own __del__ which the gc >>> will call. >> I understand. Then having a __del__ is always expensive, I guess? >> I mean, does it involve overhead to have a __del__ at all, runtime >> or compile time? > It just means there's a lot of runtime bookkeeping in the GC for > objects with __del__. > >> How feasible would it be to generate always two versions of >> the RPython class with a stub __del__ method, which calls a >> yet non-existing function? >> >> sorry if that is non-sense, but maybe something can be done >> to isolate these bad spots, without simply silently not calling them. > Well, you do at least get a nice warning. I don't think this is too > hard to work around in apps. There's also weakrefs. > Ah, we do get a warning, that is good, did not know. Anyway, the concept of RPython is pretty old, since we needed something that is treatable. And RPython is a good compromise for most things. Meanwhile, a lot of evolution has happened, most of it in the JIT area, and maybe some things can be generated more lazily and taken over by the JIT, since dynamic changes like the sudden appearance of a method could probably be handled differently than it is right now. I will shut up, this is becoming a more general topic. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From cfbolz at gmx.de Sun Feb 6 05:48:32 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 06 Feb 2011 05:48:32 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <4D4DE1EC.6020107@stackless.com> References: <4D4DCA8F.7070508@stackless.com> <4D4DD8F8.7080308@stackless.com> <4D4DE1EC.6020107@stackless.com> Message-ID: <4D4E2820.6080605@gmx.de> On Saturday, February 5, 2011, Christian Tismer wrote: > Ah, we do get a warning, that is good, did not know. > > Anyway, the concept of RPython is pretty old, since we needed > something that is treatable. And RPython is a good compromise > for most things. > > Meanwhile, a lot of evolution has happened, most of it in the JIT > area, and maybe some things can be generated more lazily > and taken over by the JIT, since dynamic changes like the sudden > appearance of a method could probably be handled differently > than it is right now. > This is not really a JIT thing at all. The GC needs to do a lot of work for every object with a del at every collection. So giving a del to all objects would need a very different approach in the GC. Cheers, Carl Friedrich From dinov at microsoft.com Sun Feb 6 06:04:37 2011 From: dinov at microsoft.com (Dino Viehland) Date: Sun, 6 Feb 2011 05:04:37 +0000 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <4D4DCA8F.7070508@stackless.com> References: <4D4DCA8F.7070508@stackless.com> Message-ID: <6C7ABA8B4E309440B857D74348836F2E0148B204@TK5EX14MBXC141.redmond.corp.microsoft.com> Christianwrote: > Howdy, > > studying the differences of PyPy vs. CPython, most seem to be fine; one > thing where I an unsure is the __del__ behavior. > > I am not addressing its delayed call or the number it is called, this is similar to > Jython and IronPython. > > But assigning to __del__ after a class is created, is that so hard to implement? IronPython also doesn't handle assigning to __del__ after the class is created, and I'd be surprised if Jython did as well. To make this work we'd need to maintain a weak reference for every object of a user defined type and I think most users would rather not pay that expense for such a corner case. I've also never actually heard of this breaking compatibility anywhere. I'd say if this was really important to you then start off w/ a nop __del__. Then you can change __del_ to whatever you want later. I'm not certain that would work w/ PyPy but I'd be surprised if it didn't - it will work w/ IronPython. From alex.gaynor at gmail.com Sun Feb 6 06:26:19 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 6 Feb 2011 00:26:19 -0500 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <6C7ABA8B4E309440B857D74348836F2E0148B204@TK5EX14MBXC141.redmond.corp.microsoft.com> References: <4D4DCA8F.7070508@stackless.com> <6C7ABA8B4E309440B857D74348836F2E0148B204@TK5EX14MBXC141.redmond.corp.microsoft.com> Message-ID: On Sun, Feb 6, 2011 at 12:04 AM, Dino Viehland wrote: > > Christianwrote: > > Howdy, > > > > studying the differences of PyPy vs. CPython, most seem to be fine; one > > thing where I an unsure is the __del__ behavior. > > > > I am not addressing its delayed call or the number it is called, this is > similar to > > Jython and IronPython. > > > > But assigning to __del__ after a class is created, is that so hard to > implement? > > IronPython also doesn't handle assigning to __del__ after the class is > created, and > I'd be surprised if Jython did as well. To make this work we'd need to > maintain a > weak reference for every object of a user defined type and I think most > users > would rather not pay that expense for such a corner case. I've also never > actually > heard of this breaking compatibility anywhere. > > I'd say if this was really important to you then start off w/ a nop > __del__. Then you can > change __del_ to whatever you want later. I'm not certain that would work > w/ > PyPy but I'd be surprised if it didn't - it will work w/ IronPython. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > I think it'll work with pypy, *and* still trigger the warning, but that's from memory. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sun Feb 6 12:57:06 2011 From: arigo at tunes.org (Armin Rigo) Date: Sun, 6 Feb 2011 12:57:06 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: References: <4D4DCA8F.7070508@stackless.com> <6C7ABA8B4E309440B857D74348836F2E0148B204@TK5EX14MBXC141.redmond.corp.microsoft.com> Message-ID: Hi Alex, On Sun, Feb 6, 2011 at 6:26 AM, Alex Gaynor wrote: >> Then you can change __del_ to whatever you want later. > > I think it'll work with pypy, *and* still trigger the warning, but that's > from memory. Just try it out then :-) The answer is no. You get a warning only if a '__del__' entry is put in a class which didn't have one so far. Armin From hakan at debian.org Sun Feb 6 17:15:16 2011 From: hakan at debian.org (Hakan Ardo) Date: Sun, 6 Feb 2011 17:15:16 +0100 Subject: [pypy-dev] Virtuals discoverd by bridge Message-ID: Hi, at the sprint we talked about how to better handle boxes that become virtual in a bridge. In the jit-virtual_state branch there is now a first step towards that. Try: py.test test/test_virtual.py -k test_dual_counter --viewloops in pypy/jit/metainterp. The idea is compare the state of all the OptValues of the jump_args at the end of the bridge white all the OptValues of the jump_args at the end of the preamble. We will need this check anyway to be sure that it is safe to call the loop (just being able to inline the short preamble turned out not to be enough). When the states differ we have 3 options: - Force some of the virtuals to make the states match - Emitt guards to make the states match - Retrace the loop to form a new specialized version We will need some heuristic to choose between these. Currently I emit GUARD_CLASS if the boxes was of correct class during the current trace (box.value). In all other cases I retrace (which is probably too often). For the test above there are 2 virtuals, node1 and node2. There will be 3 specialized version of the loop generated: - node1 virtual, node2 allocated - node2 virtual, node1 allocated - both node1 and node2 virtuals The first 2 of these are currently generated. The last is not. For the last to be generated the retraced loop will need to inherit the virtuals from the bridge that causes it to be retraced (in the same manner as the bridges inherit the virtuals of their parent loops). -- H?kan Ard? From tismer at stackless.com Tue Feb 8 23:00:53 2011 From: tismer at stackless.com (Christian Tismer) Date: Tue, 08 Feb 2011 23:00:53 +0100 Subject: [pypy-dev] Compat. in 1.4.1 __del__ In-Reply-To: <4F9D7C9F-D3CF-445E-9594-4ABD8B69E92C@underboss.org> References: <4D4DCA8F.7070508@stackless.com> <6C7ABA8B4E309440B857D74348836F2E0148B204@TK5EX14MBXC141.redmond.corp.microsoft.com> <4F9D7C9F-D3CF-445E-9594-4ABD8B69E92C@underboss.org> Message-ID: <4D51BD15.5010607@stackless.com> On 2/8/11 9:25 PM, Philip Jenvey wrote: > On Feb 5, 2011, at 9:04 PM, Dino Viehland wrote: > >> Christianwrote: >>> Howdy, >>> >>> studying the differences of PyPy vs. CPython, most seem to be fine; one >>> thing where I an unsure is the __del__ behavior. >>> >>> I am not addressing its delayed call or the number it is called, this is similar to >>> Jython and IronPython. >>> >>> But assigning to __del__ after a class is created, is that so hard to implement? >> IronPython also doesn't handle assigning to __del__ after the class is created, and >> I'd be surprised if Jython did as well. To make this work we'd need to maintain a >> weak reference for every object of a user defined type and I think most users >> would rather not pay that expense for such a corner case. I've also never actually >> heard of this breaking compatibility anywhere. > Jython doesn't either. We've been meaning to add the same warning PyPy does when you create a __del__ after the fact. > > The feature is simply not worth the effort. > > -- > Philip Jenvey Fine with me! I just wanted to know the reasons why this is more problematic than I thought. Thanks to all who replied. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From fijall at gmail.com Wed Feb 9 07:57:32 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Wed, 9 Feb 2011 08:57:32 +0200 Subject: [pypy-dev] [pypy-svn] pypy jit-virtual_state: never retrace a loop more than 5 times In-Reply-To: <20110208184238.EDF99282BF6@codespeak.net> References: <20110208184238.EDF99282BF6@codespeak.net> Message-ID: On Tue, Feb 8, 2011 at 8:42 PM, hakanardo wrote: > Author: Hakan Ardo > Branch: jit-virtual_state > Changeset: r41714:91586ef4b9c5 > Date: 2011-02-08 19:42 +0100 > http://bitbucket.org/pypy/pypy/changeset/91586ef4b9c5/ > > Log: ? ?never retrace a loop more than 5 times Is this always a good idea? Maybe we should have a configurable parameter for that (like threshold). > > diff --git a/pypy/jit/metainterp/optimizeopt/unroll.py b/pypy/jit/metainterp/optimizeopt/unroll.py > --- a/pypy/jit/metainterp/optimizeopt/unroll.py > +++ b/pypy/jit/metainterp/optimizeopt/unroll.py > @@ -671,7 +671,10 @@ > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? "jumping to preamble instead") > ? ? ? ? ? ? ? ? ? ? ? ? ? ? self.emit_operation(op) > ? ? ? ? ? ? ? ? ? ? ? ? return > - ? ? ? ? ? ? ? ?if not self.retraced: > + ? ? ? ? ? ? ? ?retraced_count = len(short) > + ? ? ? ? ? ? ? ?if descr.failed_states: > + ? ? ? ? ? ? ? ? ? ?retraced_count += len(descr.failed_states) > + ? ? ? ? ? ? ? ?if not self.retraced and retraced_count<5: > ? ? ? ? ? ? ? ? ? ? if not descr.failed_states: > ? ? ? ? ? ? ? ? ? ? ? ? raise RetraceLoop > ? ? ? ? ? ? ? ? ? ? for failed in descr.failed_states: > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > From arigo at tunes.org Thu Feb 10 12:23:51 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 10 Feb 2011 12:23:51 +0100 Subject: [pypy-dev] Fwd: [codespeak-ann] termination of codespeak.net services end FEB 2011 In-Reply-To: <20110210084509.GE14054@trillke.net> References: <20110210084509.GE14054@trillke.net> Message-ID: Hi all, It seems that Codespeak is going down sooner rather than later. Here is a copy of the start of holger's announce: ---------- Forwarded message ---------- From: holger krekel Date: Thu, Feb 10, 2011 at 9:45 AM Subject: [codespeak-ann] termination of codespeak.net services end FEB 2011 To: codespeak-ann at codespeak.net hi codespeak.net users, (sorry if you get mail twice, i wanted to make sure ...) after 8 years of operation codespeak.net services are bound to terminate, starting ? ?END OF FEBRUARY 2011 ------------------------------------------------ As far as PyPy is concerned, we need to move somewhere else: * the mailing lists pypy-dev, pypy-svn, pypy-z (also the history, please :-) * the remainings parts that are in Subversion: at least svn/pypy/extradoc, svn/pypy/benchmarks, svn/pypy/lang, which should all be converted to Mercurial I suppose (help needed here) * the developers web site (from pypy/doc/) * the users web site (http://pypy.org/) * the buildmaster * the issue tracker * the svn/user/xxx directory for people like me that use it a lot * something else? It's not like we weren't warned plenty of times. It still looks like a lot of work. The timing is a bit bad, because I wanted to finish fixing the 2.7 test failures in time to do a PyPy 1.5 release for PyCon, but it looks a bit more unlikely right now :-( (Not too important, we can do a pre-release any time, and should do one soon anyway.) A bient?t, Armin. From lac at openend.se Thu Feb 10 12:40:38 2011 From: lac at openend.se (Laura Creighton) Date: Thu, 10 Feb 2011 12:40:38 +0100 Subject: [pypy-dev] Fwd: [codespeak-ann] termination of codespeak.net services end FEB 2011 In-Reply-To: Message from Armin Rigo of "Thu, 10 Feb 2011 12:23:51 +0100." References: <20110210084509.GE14054@trillke.net> Message-ID: <201102101140.p1ABec9U026243@theraft.openend.se> We can move things to Open End if people are interested. But there is a lot to be said for moving to a place where the whole hosting matter is somebody else's problem. Right now we seem awfully dependent on home-grown tools. Are there any that people cannot live without? Note: I am playing with Sphinx and liking it. It takes ReST as input. Georg Brandl is about to come out with a release that lets readers post notes on documentation pages, which I think would be a very neat feature to have. So'd I'd sort of like to be able to run sphinx wherever we go, and, at some point convert all of our documentation to use sphinx. Laura From stefan_ml at behnel.de Thu Feb 10 13:22:34 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 10 Feb 2011 13:22:34 +0100 Subject: [pypy-dev] Fwd: [codespeak-ann] termination of codespeak.net services end FEB 2011 In-Reply-To: <201102101140.p1ABec9U026243@theraft.openend.se> References: <20110210084509.GE14054@trillke.net> <201102101140.p1ABec9U026243@theraft.openend.se> Message-ID: Laura Creighton, 10.02.2011 12:40: > We can move things to Open End if people are interested. Hmm, a web site that's full of "Alternate HTML content should be placed here." boxes doesn't sound very inviting to me as a hosting solution. Stefan From arigo at tunes.org Thu Feb 10 13:38:23 2011 From: arigo at tunes.org (Armin Rigo) Date: Thu, 10 Feb 2011 13:38:23 +0100 Subject: [pypy-dev] Fwd: [codespeak-ann] termination of codespeak.net services end FEB 2011 In-Reply-To: <201102101140.p1ABec9U026243@theraft.openend.se> References: <20110210084509.GE14054@trillke.net> <201102101140.p1ABec9U026243@theraft.openend.se> Message-ID: Hi Laura, On Thu, Feb 10, 2011 at 12:40 PM, Laura Creighton wrote: > We can move things to Open End if people are interested. ?But there is a > lot to be said for moving to a place where the whole hosting matter is > somebody else's problem. I agree, but I still think that we will take your offer for *some* of the services I described. At least the buildmaster should be moved to Open End (I doubt there is an existing provider that supports customized buildbot masters like we need). Armin From hakan at debian.org Thu Feb 10 17:12:20 2011 From: hakan at debian.org (Hakan Ardo) Date: Thu, 10 Feb 2011 17:12:20 +0100 Subject: [pypy-dev] Resume Message-ID: Hi, the jit-virtual_state branch is getting ready to be merged. It fixes the issue in test_new_virtual_member_in_bridge. However I've added a fieldstate member to the VirtualInfo classes, which might have some memory usage impacts? Would it be possible/a good idea to integrate the new VirtualStateAdder in unroll.py more closely with the resume stuff to maybe get rid of this member to use less memory? Also, at some (later) point I would like to experiment with including NotVirtualInfo objects in the resumedata to allow a bridge to inherit information about non-virtuals from it's parent loop. -- H?kan Ard? From holger at merlinux.eu Thu Feb 10 18:01:30 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 10 Feb 2011 18:01:30 +0100 Subject: [pypy-dev] Fwd: [codespeak-ann] termination of codespeak.net services end FEB 2011 In-Reply-To: References: <20110210084509.GE14054@trillke.net> Message-ID: <20110210170130.GF14054@trillke.net> Hi Armin, for pypy things are and will remain special ... sorry for not having that this morning when i sent the general announce. I was a bit in a hurry. Anyway, there is no need to fear that pypy services suddenly vanish end february. Let's discuss options sometime on IRC. I also have some suggestions ... cheers, holger > Hi all, > > It seems that Codespeak is going down sooner rather than later. Here > is a copy of the start of holger's announce: > > ---------- Forwarded message ---------- > From: holger krekel > Date: Thu, Feb 10, 2011 at 9:45 AM > Subject: [codespeak-ann] termination of codespeak.net services end FEB 2011 > To: codespeak-ann at codespeak.net > > > hi codespeak.net users, (sorry if you get mail twice, i wanted to make sure ...) > > after 8 years of operation codespeak.net services are bound to > terminate, starting > > ? ?END OF FEBRUARY 2011 > > ------------------------------------------------ > > As far as PyPy is concerned, we need to move somewhere else: > > * the mailing lists pypy-dev, pypy-svn, pypy-z (also the history, please :-) > > * the remainings parts that are in Subversion: at least > svn/pypy/extradoc, svn/pypy/benchmarks, svn/pypy/lang, which should > all be converted to Mercurial I suppose (help needed here) > > * the developers web site (from pypy/doc/) > > * the users web site (http://pypy.org/) > > * the buildmaster > > * the issue tracker > > * the svn/user/xxx directory for people like me that use it a lot > > * something else? > > It's not like we weren't warned plenty of times. It still looks like > a lot of work. The timing is a bit bad, because I wanted to finish > fixing the 2.7 test failures in time to do a PyPy 1.5 release for > PyCon, but it looks a bit more unlikely right now :-( (Not too > important, we can do a pre-release any time, and should do one soon > anyway.) > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- From dcolish at gmail.com Sun Feb 13 02:53:16 2011 From: dcolish at gmail.com (Dan Colish) Date: Sat, 12 Feb 2011 17:53:16 -0800 Subject: [pypy-dev] PSF Sponsored Sprint in Portland, OR Message-ID: Hello Developers! I'm one of the organizers of the Portland PSF Sprint. We've been working hard on our proposal to come up with a strong program for a full day of sprinting in Portland. The day will mostly focus on porting libraries to Python 3, hacking on PyPY 2.7 compatibility and testing. It'll be a great time so if you're in the area stop on by! The current project list includes:- Fabric - Django - Flatland - PyPy - Alfajor Want to suggest your own? Use the signup sheet here. Breakfast is being sponsored by Idealist.org, lunch is being sponsored by Emma and beverages will be made available by Urban Airship. More information: http://goo.gl/XTMWv. Signup sheet: http://goo.gl/a5Gqk. -- Dan From vsapre80 at gmail.com Sun Feb 13 18:55:43 2011 From: vsapre80 at gmail.com (Vishal) Date: Sun, 13 Feb 2011 23:25:43 +0530 Subject: [pypy-dev] Hello...and a small question Message-ID: Hello, This is my first email to the PyPy Dev mailing list. I would like to ask if there is a place where a potential user of PyPy would know, what C extensions support PyPy? Could this be a way of helping PyPy? -- Thanks and best regards, Vishal Sapre --- "So say...Day by day, in every way, I am getting better, better and better !!!" "A Strong and Positive attitude creates more miracles than anything else. Because...Life is 10% how you make it, and 90% how you take it" "Diamond is another piece of coal that did well under pressure? "May we do good and not evil. May we find forgiveness for ourself and forgive others. May we share freely, never taking more than we give." -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Mon Feb 14 01:42:46 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 14 Feb 2011 00:42:46 +0000 Subject: [pypy-dev] Winter Sprint report: first draft Message-ID: Hey guys, I finally found time to write a first draft of a blog entry reporting on the winter sprint. Comments, corrections, insults and suggestions solicited: PyPy Winter Sprint A few weeks ago I had the great fortune to attend the winter sprint in Leysin Switzerland. I've wanted to contribute to PyPy for a long time and I thought diving into a sprint might be a good way to get familiar with some of the code. What I wasn't expecting was to be implementing new built-in methods in RPython on the first day. The main thing I took away from the sprint was just how easy it is to get involved in developing PyPy (well, some bits of it at least and being surrounded by core developers helps). I wrote up a very short description of how to get started here ( https://bitbucket.org/pypy/pypy/wiki/How%20to%20run%20lib-python%20tests ), but I'll do a longer blog post with examples on my own blog ( http://www.voidspace.org.uk/python/weblog/ ) soon(ish). The sprint was kicked off by Armin merging the "fast-forward" branch of PyPy onto trunk. "fast-forward" brings PyPy from Python 2.5 compatibility to Python 2.7. Along with this it brought a large number of test failures, as the sterling work done by Benjamin Peterson and Amaury Forgeot d'Arc was not complete. This immediately set the primary sprint goal to reduce the number of test failures. We made a great deal of progress on this front, and you can see how close PyPy is now from the buildbot: http://codespeak.net:8099/summary?branch=%3Ctrunk%3E Jacob Hallen and I started working through the list alphabetically. We made short work of test_asyncore and moved onto test_bytes where I was stuck for the rest of the sprint. I spent much of the remaining days working with Laura Creighton on the pypy bytearray implementation to make it more compatible with the Python 2.7 implementation. This meant adding new methods, changing some of the Python protocol method implementations and even changing the way that bytearray is constructed. All in all great fun and a great introduction to working with RPython. If you add on top of this the great people, the beautiful scenery, the Swiss cheese fondues, managing to not kill myself with a days skiing and traditional pypy card games, I can heartily recommend pypy sprints as a close approximation of geek nirvana. Working on 2.7 compatibility wasn't the only work that happened during the sprint. Other activities included: * Antonio Cuni worked on the "jittypes" branch. This is a reimplementation of the core of the PyPy ctypes code to make it jittable. The goal is that for common cases the jit should be able to turn ctypes calls into direct jumps. This work was not completed but very close and is great for the future of integrating C libraries with PyPy. As ctypes is also available in CPython and IronPython, and hopefully will be available in Jython soon, integrating C code with Python through ctypes is the most "implementation portable" technique. * David Schneider continued his work on the JIT backend for ARM. PyPy has been cross-compilable to ARM for a long time, but bringing the JIT to ARM will provide a *fast* PyPy for ARM, which includes platforms like Android. Again David didn't complete this work but did complete the float support. * Hakan Ardo was present for two days and continued his crazy-clever work on JIT optimisations, some of which are described in the "Loop invariant code motion" ( http://morepypy.blogspot.com/2011/01/loop-invariant-code-motion.html ) blog entry. * Holger Krekel worked on updating the PyPy test suite to the latest version of py.test and also worked with me on the interminable bytearray changes for part of the sprint. * No one was sure what Maciej Fijalkowsk worked on but he seemed to be quite busy. I think that was most of the work done during the actual sprint. There was also a great deal of healthy discussion about the future of PyPy. Expect lots more interesting and exciting developments over the coming year. Michael Foord -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Mon Feb 14 06:04:52 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Mon, 14 Feb 2011 00:04:52 -0500 Subject: [pypy-dev] Hello...and a small question In-Reply-To: References: Message-ID: On Sun, Feb 13, 2011 at 12:55 PM, Vishal wrote: > Hello, > > This is my first email to the PyPy Dev mailing list. > > I would like to ask if there is a place where a potential user of PyPy > would know, what C extensions support PyPy? > > Could this be a way of helping PyPy? > > -- > Thanks and best regards, > Vishal Sapre > > --- > "So say...Day by day, in every way, I am getting better, better and better > !!!" > "A Strong and Positive attitude creates more miracles than anything else. > Because...Life is 10% how you make it, and 90% how you take it" > "Diamond is another piece of coal that did well under pressure? > "May we do good and not evil. May we find forgiveness for ourself and > forgive others. May we share freely, never taking more than we give." > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > Someone started a wikipage on this subject: https://bitbucket.org/pypy/compatibility/wiki/Home, help filling it out is always welcome. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Mon Feb 14 06:11:44 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 13 Feb 2011 22:11:44 -0700 Subject: [pypy-dev] Winter Sprint report: first draft In-Reply-To: References: Message-ID: Hey. Few comments: * jittypes - it's technically called C-level calls and not jumps. So ctypes calls become direct calls on the C level. Jumps are without calling mechanism. * My name is missing an "i" at the end. Also unicode friendly name is Fija?kowski (although it's fine to keep it that way to remain british and keep cultural heritage of horrible misspelling). * I was working on investigating why twisted_tcp got slower and shaving yaks in between, but what you said pretty much describes what I did, even in a better detail. On Sun, Feb 13, 2011 at 5:42 PM, Michael Foord wrote: > Hey guys, > I finally found time to write a first draft of a blog entry reporting on the > winter sprint. Comments, corrections, insults and suggestions solicited: > > PyPy Winter Sprint > A few weeks ago I had the great fortune to attend the winter sprint in > Leysin Switzerland. I've wanted to contribute to PyPy for a long time and I > thought diving into a sprint might be a good way to get familiar with some > of the code. What I wasn't expecting was to be implementing new built-in > methods in RPython on the first day. The main thing I took away from the > sprint was just how easy it is to get involved in developing PyPy (well, > some bits of it at least and being surrounded by core developers helps). I > wrote up a very short description of how to get started here ( > https://bitbucket.org/pypy/pypy/wiki/How%20to%20run%20lib-python%20tests ), > but I'll do a longer blog post with examples on my own blog ( > http://www.voidspace.org.uk/python/weblog/ ) soon(ish). > The sprint was kicked off by Armin merging the "fast-forward" branch of PyPy > onto trunk. "fast-forward" brings PyPy from Python 2.5 compatibility to > Python 2.7. Along with this it brought a large number of test failures, as > the sterling work done by Benjamin Peterson and Amaury Forgeot d'Arc was not > complete. This immediately set the primary sprint goal to reduce the number > of test failures. > We made a great deal of progress on this front, and you can see how close > PyPy is now from the buildbot: > http://codespeak.net:8099/summary?branch=%3Ctrunk%3E > Jacob Hallen and I started working through the list alphabetically. We made > short work of test_asyncore and moved onto test_bytes where I was stuck for > the rest of the sprint. I spent much of the remaining days working with > Laura Creighton on the pypy bytearray implementation to make it more > compatible with the Python 2.7 implementation. This meant adding new > methods, changing some of the Python protocol method implementations and > even changing the way that bytearray is constructed. All in all great fun > and a great introduction to working with RPython. > If you add on top of this the great people, the beautiful scenery, the Swiss > cheese fondues, managing to not kill myself with a days skiing and > traditional pypy card games, I can heartily recommend pypy sprints as a > close approximation of geek nirvana. > Working on 2.7 compatibility wasn't the only work that happened during the > sprint. Other activities included: > * Antonio Cuni worked on the "jittypes" branch. This is a reimplementation > of the core of the PyPy ctypes code to make it jittable. The goal is that > for common cases the jit should be able to turn ctypes calls into direct > jumps. This work was not completed but very close and is great for the > future of integrating C libraries with PyPy. As ctypes is also available in > CPython and IronPython, and hopefully will be available in Jython soon, > integrating C code with Python through ctypes is the most "implementation > portable" technique. > * David Schneider continued his work on the JIT backend for ARM. PyPy has > been cross-compilable to ARM for a long time, but bringing the JIT to ARM > will provide a *fast* PyPy for ARM, which includes platforms like Android. > Again David didn't complete this work but did complete the float support. > * Hakan Ardo was present for two days and continued his crazy-clever work on > JIT optimisations, some of which are described in the "Loop invariant code > motion" ( > http://morepypy.blogspot.com/2011/01/loop-invariant-code-motion.html ) blog > entry. > * Holger Krekel worked on updating the PyPy test suite to the latest version > of py.test and also worked with me on the interminable bytearray changes for > part of the sprint. > * No one was sure what ?Maciej Fijalkowsk worked on but he seemed to be > quite busy. > I think that was most of the work done during the actual sprint. There was > also a great deal of healthy discussion about the future of PyPy. Expect > lots more interesting and exciting developments over the coming year. > Michael Foord > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From dimaqq at gmail.com Mon Feb 14 06:20:30 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Sun, 13 Feb 2011 22:20:30 -0700 Subject: [pypy-dev] Winter Sprint report: first draft In-Reply-To: References: Message-ID: On 13 February 2011 22:11, Maciej Fijalkowski wrote: > * My name is missing an "i" at the end. Also unicode friendly name is > Fija?kowski (although it's fine to keep it that way to remain british > and keep cultural heritage of horrible misspelling). In the interest of the Commonwealth you are from here on Matthew Violet ;-) d. From arigo at tunes.org Mon Feb 14 11:36:22 2011 From: arigo at tunes.org (Armin Rigo) Date: Mon, 14 Feb 2011 11:36:22 +0100 Subject: [pypy-dev] Winter Sprint report: first draft In-Reply-To: References: Message-ID: Hi Michael, Thanks for the upcoming blog entry! :-) On Mon, Feb 14, 2011 at 1:42 AM, Michael Foord wrote: > http://codespeak.net:8099/summary?branch=%3Ctrunk%3E It's best to point to the future-proofer url: http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E . Armin From fuzzyman at gmail.com Mon Feb 14 11:40:12 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 14 Feb 2011 10:40:12 +0000 Subject: [pypy-dev] Winter Sprint report: first draft In-Reply-To: References: Message-ID: Thanks for the feedback and corrections folks. Either someone else will have to post the slightly edited version, or someone will need to give me permissions to add blog entries. (For blogger *this* emil address is my blogger id I'm pretty sure.) Either way, maybe worth waiting a little bit in case any of the other folk from the sprint want to correct anything else. All the best, Michael On 14 February 2011 10:36, Armin Rigo wrote: > Hi Michael, > > Thanks for the upcoming blog entry! :-) > > On Mon, Feb 14, 2011 at 1:42 AM, Michael Foord wrote: > > http://codespeak.net:8099/summary?branch=%3Ctrunk%3E > > It's best to point to the future-proofer url: > http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E . > > > Armin > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at gmail.com Mon Feb 14 13:06:30 2011 From: fuzzyman at gmail.com (Michael Foord) Date: Mon, 14 Feb 2011 12:06:30 +0000 Subject: [pypy-dev] Winter Sprint report: first draft In-Reply-To: References: Message-ID: It's now live, but I can still edit it and correct mistakes of course: http://morepypy.blogspot.com/2011/02/pypy-winter-sprint-report.html Thanks for a great sprint everyone! Michael On 14 February 2011 10:40, Michael Foord wrote: > Thanks for the feedback and corrections folks. Either someone else will > have to post the slightly edited version, or someone will need to give me > permissions to add blog entries. (For blogger *this* emil address is my > blogger id I'm pretty sure.) > > Either way, maybe worth waiting a little bit in case any of the other folk > from the sprint want to correct anything else. > > All the best, > > Michael > > > On 14 February 2011 10:36, Armin Rigo wrote: > >> Hi Michael, >> >> Thanks for the upcoming blog entry! :-) >> >> On Mon, Feb 14, 2011 at 1:42 AM, Michael Foord >> wrote: >> > http://codespeak.net:8099/summary?branch=%3Ctrunk%3E >> >> It's best to point to the future-proofer url: >> http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E . >> >> >> Armin >> > > > > -- > > http://www.voidspace.org.uk/ > > May you do good and not evil > May you find forgiveness for yourself and forgive others > > May you share freely, never taking more than you give. > -- the sqlite blessing http://www.sqlite.org/different.html > > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From victorgarcianet at gmail.com Mon Feb 14 23:41:09 2011 From: victorgarcianet at gmail.com (Victor Garcia) Date: Mon, 14 Feb 2011 20:41:09 -0200 Subject: [pypy-dev] PyPy Compatibility Wiki Message-ID: Hi Vishal (and list), Thanks for wanting to help PyPy :) I'm working on the compatibility wiki and I like your idea of having a page like http://getpython3.net/ to summarize it and make it prettier. We could have it as a totally separated site (hosting needed, any implementation we want), a separated site hosted on bitbucket (like getpypy.bitbucket.org, static HTML + JS needed, any design we want) or as the front page of the wiki. The whole design of the wiki is open to changes. The information from the wiki is available in YAML format (there's a script to convert YAML<->creole and do other checks), so it would be very easy to generate a different format for a site like that. Right now I'd like to invest what time I have to improve coverage of compatibility-tested packages and implement some items listed in https://bitbucket.org/pypy/compatibility/wiki/Discussion#!to-do, but I'd sure help with building a site base on wiki info if people agree it's a good idea. Since we haven't discussed the wiki much at all, I'd also like to invite everyone to chime in about what they'd like to see done in the wiki. Victor -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Tue Feb 15 02:13:54 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 14 Feb 2011 18:13:54 -0700 Subject: [pypy-dev] Separate compilation and friends Message-ID: Hi. There is growing interest about PyPy and especially about extension modules. Apparently there are some people (like Alex) that are willing to write modules in RPython that should not go to the main tree. Since separate compilation is considered hard, how hard would it be to provide separate loading? This would mean you still compile the whole interpreter, but you can load the module from a compiled PyPy that didn't have this option on. Cheers, fijal From dimaqq at gmail.com Tue Feb 15 03:41:11 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Mon, 14 Feb 2011 19:41:11 -0700 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: Message-ID: On a related note, how hard is it to "freeze" the translator/compiler state of a given pypy version just before it begins to read extension modules and distribute that, it would speed up module development a lot. It would be a quick equivalent of distributing headers for a C library. d. On 14 February 2011 18:13, Maciej Fijalkowski wrote: > Hi. > > There is growing interest about PyPy and especially about extension > modules. Apparently there are some people (like Alex) that are willing > to write modules in RPython that should not go to the main tree. Since > separate compilation is considered hard, how hard would it be to > provide separate loading? This would mean you still compile the whole > interpreter, but you can load the module from a compiled PyPy that > didn't have this option on. > > Cheers, > fijal > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From william.leslie.ttg at gmail.com Tue Feb 15 04:34:44 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Tue, 15 Feb 2011 14:34:44 +1100 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: Message-ID: On 15 February 2011 12:13, Maciej Fijalkowski wrote: > Hi. > > There is growing interest about PyPy and especially about extension > modules. Apparently there are some people (like Alex) that are willing > to write modules in RPython that should not go to the main tree. Since > separate compilation is considered hard, how hard would it be to > provide separate loading? This would mean you still compile the whole > interpreter, but you can load the module from a compiled PyPy that > didn't have this option on. By "this option" you mean --withmod-xxx ? -- William Leslie From anto.cuni at gmail.com Tue Feb 15 09:10:08 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 15 Feb 2011 09:10:08 +0100 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: Message-ID: <4D5A34E0.3060602@gmail.com> On 15/02/11 03:41, Dima Tisnek wrote: > On a related note, how hard is it to "freeze" the translator/compiler > state of a given pypy version just before it begins to read extension > modules and distribute that, it would speed up module development a > lot. > It would be a quick equivalent of distributing headers for a C library. this is hard, because the compilation of the modules is intermixed with the compilation of the rest of the interpreter for each phase: we have (roughly) something like: - annotation of the interpreter - annotation of the modules - rtyping of the interpreter - rtyping of the modules - etc. etc. ciao, Anto From dimaqq at gmail.com Tue Feb 15 09:17:58 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 15 Feb 2011 01:17:58 -0700 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: <4D5A34E0.3060602@gmail.com> References: <4D5A34E0.3060602@gmail.com> Message-ID: On 15 February 2011 01:10, Antonio Cuni wrote: > On 15/02/11 03:41, Dima Tisnek wrote: >> On a related note, how hard is it to "freeze" the translator/compiler >> state of a given pypy version just before it begins to read extension >> modules and distribute that, it would speed up module development a >> lot. >> It would be a quick equivalent of distributing headers for a C library. > > this is hard, because the compilation of the modules is intermixed with the > compilation of the rest of the interpreter for each phase: we have (roughly) > something like: > > - annotation of the interpreter > - annotation of the modules > - rtyping of the interpreter > - rtyping of the modules > - etc. etc. > > ciao, > Anto > Yeah I figured as much, I was wondering if it could be changed like this: - annotation of the interpreter, save state (1) - rtyping of the interpreter, shouldn't depend on modules here, save state (2) - etc. - annotation of the modules, using state from 1 - rtyping of the modules, using state from 1,2 - etc. I assume here that modules don't introduce dependencies into the iterpreter. I guess in the long run this ought to be the case, right? If this is possible, it would be a useful quick hack to separate module build from main build. If it's still very hard, then some else is in order. I'd love to play with this myself, but I don't have enough ram for a full build ;-( d. From pgiarrusso at mathematik.uni-marburg.de Tue Feb 15 09:21:05 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Tue, 15 Feb 2011 09:21:05 +0100 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: <4D5A34E0.3060602@gmail.com> References: <4D5A34E0.3060602@gmail.com> Message-ID: On Tue, Feb 15, 2011 at 09:10, Antonio Cuni wrote: > On 15/02/11 03:41, Dima Tisnek wrote: >> On a related note, how hard is it to "freeze" the translator/compiler >> state of a given pypy version just before it begins to read extension >> modules and distribute that, it would speed up module development a >> lot. >> It would be a quick equivalent of distributing headers for a C library. > > this is hard, because the compilation of the modules is intermixed with the > compilation of the rest of the interpreter for each phase: we have (roughly) > something like: > > - annotation of the interpreter > - annotation of the modules > - rtyping of the interpreter > - rtyping of the modules > - etc. etc. Thanks for explaining the problem. IIRC, rtyping is a global analysis because for performance you perform type inference globally, so that for instance if a module allows inferring that a field is a number, that information is propagated to the original definition. Depending on what Maciej meant, his proposal might be as hard: > [Separate loading] This would mean you still compile the whole interpreter, but you can load the module from a compiled PyPy that didn't have this option on. At least as far as I understand, if "interpreter" means just the interpreter "core", as in Antonio's mail, this seems just as hard. Otherwise, if "interpreter" includes modules, this extension would not accomodate the needs of authors of external modules: one would have download all external modules and compile them together, but at runtime one could save the RAM footprint of external modules. Is that the relevant problem? Cheers, -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From pgiarrusso at mathematik.uni-marburg.de Tue Feb 15 09:32:50 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Tue, 15 Feb 2011 09:32:50 +0100 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: <4D5A34E0.3060602@gmail.com> Message-ID: On Tue, Feb 15, 2011 at 09:17, Dima Tisnek wrote: > On 15 February 2011 01:10, Antonio Cuni wrote: >> On 15/02/11 03:41, Dima Tisnek wrote: >>> On a related note, how hard is it to "freeze" the translator/compiler >>> state of a given pypy version just before it begins to read extension >>> modules and distribute that, it would speed up module development a >>> lot. >>> It would be a quick equivalent of distributing headers for a C library. >> >> this is hard, because the compilation of the modules is intermixed with the >> compilation of the rest of the interpreter for each phase: we have (roughly) >> something like: >> >> - annotation of the interpreter >> - annotation of the modules How much time would be saved by saving the annotation results? >> - rtyping of the interpreter >> - rtyping of the modules >> - etc. etc. >> >> ciao, >> Anto >> > > Yeah I figured as much, I was wondering if it could be changed like this: > > - annotation of the interpreter, save state (1) > - rtyping of the interpreter, shouldn't depend on modules here, save state (2) > - etc. > - annotation of the modules, using state from 1 > - rtyping of the modules, using state from 1,2 > - etc. > > I assume here that modules don't introduce dependencies into the iterpreter. > I guess in the long run this ought to be the case, right? I don't think you can guarantee this. Type inference is global, and you might need a user for each API to better infer its type. Maybe uses of an API in testcases allow fully inferring their types, but I'd guess not. However, what is true in general is that if less specific types are inferred, that affects just performance, not correctness (I don't know if that's true of PyPy, but you ought to be able to pass "object"s around). Maybe the slowdown is insignificant, maybe it is a huge problem, maybe few annotations can save the day. However, it is still not clear (to me) where previous efforts stopped. Is it hard to: 1) devise an algorithm like Dima proposed or to 2) implement it (because of too much code to change and limited manpower) or to 3) or to have a small performance loss? Per-file separate compilation would likely fall into 3), because too little type inference would happen, isn't it? > If this is possible, it would be a useful quick hack to separate > module build from main build. > If it's still very hard, then some else is in order. > I'd love to play with this myself, but I don't have enough ram for a > full build ;-( -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From dimaqq at gmail.com Tue Feb 15 10:01:21 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 15 Feb 2011 02:01:21 -0700 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: <4D5A34E0.3060602@gmail.com> Message-ID: On 15 February 2011 01:32, Paolo Giarrusso wrote: > On Tue, Feb 15, 2011 at 09:17, Dima Tisnek wrote: >> On 15 February 2011 01:10, Antonio Cuni wrote: >>> On 15/02/11 03:41, Dima Tisnek wrote: >>>> On a related note, how hard is it to "freeze" the translator/compiler >>>> state of a given pypy version just before it begins to read extension >>>> modules and distribute that, it would speed up module development a >>>> lot. >>>> It would be a quick equivalent of distributing headers for a C library. >>> >>> this is hard, because the compilation of the modules is intermixed with the >>> compilation of the rest of the interpreter for each phase: we have (roughly) >>> something like: >>> >>> - annotation of the interpreter >>> - annotation of the modules > > How much time would be saved by saving the annotation results? > >>> - rtyping of the interpreter >>> - rtyping of the modules >>> - etc. etc. >>> >>> ciao, >>> Anto >>> >> >> Yeah I figured as much, I was wondering if it could be changed like this: >> >> - annotation of the interpreter, save state (1) >> - rtyping of the interpreter, shouldn't depend on modules here, save state (2) >> - etc. >> - annotation of the modules, using state from 1 >> - rtyping of the modules, using state from 1,2 >> - etc. >> >> I assume here that modules don't introduce dependencies into the iterpreter. >> I guess in the long run this ought to be the case, right? > > I don't think you can guarantee this. Type inference is global, and > you might need a user for each API to better infer its type. Maybe > uses of an API in testcases allow fully inferring their types, but I'd > guess not. > > However, what is true in general is that if less specific types are > inferred, that affects just performance, not correctness (I don't know > if that's true of PyPy, but you ought to be able to pass "object"s > around). Maybe the slowdown is insignificant, maybe it is a huge > problem, maybe few annotations can save the day. Correct me if I'm wong, but I assume that rpython and python type inferences are quite different. After all we don't have user code when we translate pypy itself, do we? Coming back to rpython-only discussion: I suppose one option woud be include type speciation in the modules themselves, or is that an overkill? Say if I write an ubermodule with uberdatum class in rpython and then use standard library bisect in that module, currenty as all modules and core are compiled together, an uberdatum-specific bisect loop can be made. If we move ubermodule out, it cannot be done. At the same time, if modules carried own versions of everything including all builtins, those modules would be just too big. Something in between would be a fixed "ABI" where core and stdlib are one blob and each 3rd party module is another blob, and everything that crosses the boundary is treated as a "pyobject". > > However, it is still not clear (to me) where previous efforts stopped. > Is it hard to: > 1) devise an algorithm like Dima proposed > or to > 2) implement it (because of too much code to change and limited manpower) > or to > 3) or to have a small performance loss? > > Per-file separate compilation would likely fall into 3), because too > little type inference would happen, isn't it? > >> If this is possible, it would be a useful quick hack to separate >> module build from main build. >> If it's still very hard, then some else is in order. >> I'd love to play with this myself, but I don't have enough ram for a >> full build ;-( > > -- > Paolo Giarrusso - Ph.D. Student > http://www.informatik.uni-marburg.de/~pgiarrusso/ > From mail at stevesimmons.com Tue Feb 15 09:57:21 2011 From: mail at stevesimmons.com (Stephen Simmons) Date: Tue, 15 Feb 2011 02:57:21 -0600 (CST) Subject: [pypy-dev] European sprints? Message-ID: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> Hi... I have just started following the PyPy mailing list and am interested in getting involved in some small way. Are there any European sprints planned in the coming few months? Thanks Stephen From holger at merlinux.eu Tue Feb 15 11:25:06 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 15 Feb 2011 11:25:06 +0100 Subject: [pypy-dev] google should build a pypy phone or someone Message-ID: <20110215102506.GM30557@trillke.net> Hi all, what do you think or know why Google choose Java over Python for its Android system? Surely, Python doesn't have a big track record in terms of targetting mobile devices. Also CPython is limited in how it can change and adapt i guess. I imagine PyPy could do much better. Wouldn't it be cool to have a totally python-based state-of-the-art phone? :) cheers, holger From anto.cuni at gmail.com Tue Feb 15 11:36:43 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 15 Feb 2011 11:36:43 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <20110215102506.GM30557@trillke.net> References: <20110215102506.GM30557@trillke.net> Message-ID: <4D5A573B.40403@gmail.com> On 15/02/11 11:25, holger krekel wrote: > Hi all, > > what do you think or know why Google choose Java over Python for > its Android system? Surely, Python doesn't have a big track record > in terms of targetting mobile devices. Also CPython is limited > in how it can change and adapt i guess. I imagine PyPy could do much > better. Wouldn't it be cool to have a totally python-based > state-of-the-art phone? :) yes: this way, when my friends ask me what the hell I'm working on, I could just show them the phone :-) More seriously, I think that nowadays one of the most important points for a mobile platform is to have a huge selection of 3rd party apps: in this sense, the popularity of Java and the big number of developers (especially mediocre and cheap developers, which is what you need to develop most of the apps out there) is a big win for android. On the other hand, iphone shows that you can do that even if your primary language is Objective-C, but it AFAIK objc was already used quite a lot in the apple ecosystem. Anyway, back to the original topic: what would python offer more than java for android? ciao, Anto From chef at ghum.de Tue Feb 15 11:52:23 2011 From: chef at ghum.de (Massa, Harald Armin) Date: Tue, 15 Feb 2011 11:52:23 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <4D5A573B.40403@gmail.com> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> Message-ID: Antonio, > > Anyway, back to the original topic: what would python offer more than java > for > android? > > a much less challenging licence-situation. Harald -- GHUM GmbH Harald Armin Massa Spielberger Stra?e 49 70435 Stuttgart 0173/9409607 Amtsgericht Stuttgart, HRB 734971 - persuadere. et programmare -------------- next part -------------- An HTML attachment was scrubbed... URL: From lac at openend.se Tue Feb 15 11:54:35 2011 From: lac at openend.se (Laura Creighton) Date: Tue, 15 Feb 2011 11:54:35 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: Message from "Stephen Simmons" of "Tue, 15 Feb 2011 02:57:21 CST." <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> Message-ID: <201102151054.p1FAsZWb018955@theraft.openend.se> In a message of Tue, 15 Feb 2011 02:57:21 CST, "Stephen Simmons" writes: >Hi... > >I have just started following the PyPy mailing list and am interested in >getting involved in some small way. > >Are there any European sprints planned in the coming few months? > >Thanks >Stephen I want to have one in G?teborg some time after PyCON. That's about as far as this discussion has got so far. Laura From holger at merlinux.eu Tue Feb 15 11:55:42 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 15 Feb 2011 11:55:42 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> Message-ID: <20110215105542.GN30557@trillke.net> On Tue, Feb 15, 2011 at 11:52 +0100, Massa, Harald Armin wrote: > Antonio, > > > > > Anyway, back to the original topic: what would python offer more than java > > for > > android? > > > > a much less challenging licence-situation. > > Harald for those not in the know: Oracle filed a patents lawsuit against Googles Android Dalvik VM which is maybe mostly but not completely without merits. Holger > > -- > GHUM GmbH > Harald Armin Massa > Spielberger Stra?e 49 > 70435 Stuttgart > 0173/9409607 > > Amtsgericht Stuttgart, HRB 734971 > - > persuadere. > et programmare > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- From chef at ghum.de Tue Feb 15 12:00:31 2011 From: chef at ghum.de (Massa, Harald Armin) Date: Tue, 15 Feb 2011 12:00:31 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <20110215105542.GN30557@trillke.net> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <20110215105542.GN30557@trillke.net> Message-ID: > > > > > a much less challenging licence-situation. > > for those not in the know: Oracle filed a patents lawsuit against Googles > Android > Dalvik VM which is maybe mostly but not completely without merits. > and, independend of the merits that may or may not be present, the legal expenses and collateral damage of that lawsuit will most surely be higher than the money put into PyPy the last 10years, European funding included. Harald -- GHUM GmbH Harald Armin Massa Spielberger Stra?e 49 70435 Stuttgart 0173/9409607 Amtsgericht Stuttgart, HRB 734971 - persuadere. et programmare -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Tue Feb 15 12:14:29 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 15 Feb 2011 12:14:29 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> Message-ID: <4D5A6015.9010509@gmail.com> On 15/02/11 11:52, Massa, Harald Armin wrote: > Antonio, > > > Anyway, back to the original topic: what would python offer more than java for > android? > > a much less challenging licence-situation. I'm not completely sure. AFAIK, the Oracle-Google lawsuit is about patents, not licences: in particular, they are complaining about how Dalvik is implemented, not about the fact that it implements Java. >From what I could read at least a couple of those patents apply to PyPy as well (and to whatever language with JIT, fwiw), e.g. IIRC there is one about "compiling an optimized version of code that executes often" or something similar (whether this patent is valid is another topic, of course). ciao, Anto From holger at merlinux.eu Tue Feb 15 12:26:56 2011 From: holger at merlinux.eu (holger krekel) Date: Tue, 15 Feb 2011 12:26:56 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <4D5A6015.9010509@gmail.com> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <4D5A6015.9010509@gmail.com> Message-ID: <20110215112656.GO30557@trillke.net> On Tue, Feb 15, 2011 at 12:14 +0100, Antonio Cuni wrote: > On 15/02/11 11:52, Massa, Harald Armin wrote: > > Antonio, > > > > > > Anyway, back to the original topic: what would python offer more than java for > > android? > > > > a much less challenging licence-situation. > > I'm not completely sure. > AFAIK, the Oracle-Google lawsuit is about patents, not licences: in > particular, they are complaining about how Dalvik is implemented, not about > the fact that it implements Java. > > >From what I could read at least a couple of those patents apply to PyPy as > well (and to whatever language with JIT, fwiw), e.g. IIRC there is one about > "compiling an optimized version of code that executes often" or something > similar (whether this patent is valid is another topic, of course). FWIW, I haven't read that patent and i doubt Armin who invented Psyco ages ago and many parts of PyPy, has. Or other people inviting JIT-compilers since 1970 for that matter :) Being safe from patents is probably impossible, at least in the US. Still the situation with PyPy should be better as we didn't start with code that had patents already mentioned left and right. But let's try to not discuss patents and licensing too much. I am fine with it being a potentially pro-pypy aspect :) best, holger From lac at openend.se Tue Feb 15 12:51:49 2011 From: lac at openend.se (Laura Creighton) Date: Tue, 15 Feb 2011 12:51:49 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: Message from Antonio Cuni of "Tue, 15 Feb 2011 11:36:43 +0100." <4D5A573B.40403@gmail.com> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> Message-ID: <201102151151.p1FBpnsk024534@theraft.openend.se> In a message of Tue, 15 Feb 2011 11:36:43 +0100, Antonio Cuni writes: > >Anyway, back to the original topic: what would python offer more than jav >a for >android? > >ciao, >Anto As some of you saw in Leysin, I own one of these. http://www.freedominput.com/freedom-accessories/freedom-pro-keyboard which works with Android. which makes the whole idea of being able to sit down and program whenever the mood strikes you, even when your laptop is far away or out of power more attractive, since you can touchtype. But that doesn't make me want to start programming in java. Especially for the sort of quick and dirty calculations I use python -c for. Laura From cfbolz at gmx.de Tue Feb 15 15:02:28 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 15 Feb 2011 15:02:28 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: <201102151054.p1FAsZWb018955@theraft.openend.se> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> Message-ID: <4D5A8774.3050700@gmx.de> On 02/15/2011 11:54 AM, Laura Creighton wrote: > In a message of Tue, 15 Feb 2011 02:57:21 CST, "Stephen Simmons" writes: >> Hi... >> >> I have just started following the PyPy mailing list and am interested in >> getting involved in some small way. >> >> Are there any European sprints planned in the coming few months? >> >> Thanks >> Stephen > > I want to have one in G?teborg some time after PyCON. That's about > as far as this discussion has got so far. Also I guess it is likely that we will have another one in D?sseldorf later in the year. Carl Friedrich From cfbolz at gmx.de Tue Feb 15 15:06:07 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 15 Feb 2011 15:06:07 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <201102151151.p1FBpnsk024534@theraft.openend.se> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <201102151151.p1FBpnsk024534@theraft.openend.se> Message-ID: <4D5A884F.9020405@gmx.de> On 02/15/2011 12:51 PM, Laura Creighton wrote: > In a message of Tue, 15 Feb 2011 11:36:43 +0100, Antonio Cuni writes: > >> >> Anyway, back to the original topic: what would python offer more than jav >> a for >> android? >> >> ciao, >> Anto > > As some of you saw in Leysin, I own one of these. > http://www.freedominput.com/freedom-accessories/freedom-pro-keyboard which > works with Android. > > which makes the whole idea of being able to sit down and program whenever > the mood strikes you, even when your laptop is far away or out of power > more attractive, since you can touchtype. > > But that doesn't make me want to start programming in java. > Especially for the sort of quick and dirty calculations I use python > -c for. I am sure you know of this then: http://code.google.com/p/android-scripting/ It's an app for Android that lets you script the phone with various dynamic languages (including Python). Actually I think that PyPy could/should be made to work with this thing. It would not quite make PyPy a first class VM on the phone, but would already give you access to a number of phone-specific features. Carl Friedrich From anto.cuni at gmail.com Tue Feb 15 15:06:50 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Tue, 15 Feb 2011 15:06:50 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: <4D5A8774.3050700@gmx.de> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> <4D5A8774.3050700@gmx.de> Message-ID: <4D5A887A.2040206@gmail.com> On 15/02/11 15:02, Carl Friedrich Bolz wrote: > Also I guess it is likely that we will have another one in D?sseldorf > later in the year. and probably one after europython, as usual? ciao, Anto From renesd at gmail.com Tue Feb 15 15:16:25 2011 From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=) Date: Tue, 15 Feb 2011 14:16:25 +0000 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <20110215102506.GM30557@trillke.net> References: <20110215102506.GM30557@trillke.net> Message-ID: Hi, a pypy phone would be lovely :) Especially if it could take advantage of the graphics processor (OpenGL ES2 shader language), and the programmable DSP chips (eg OMAP3). btw, python runs on nokia phones (with nokia supplied python, s60 and mamaeo/meego), windows CE phones/pocket pcs, android (see recent pygame android release), and even apple phones (it used to be technically banned from app store, but has run on i* for a long time. I guess the main "python platform" is OLPC, which is still quite interesting. ps. ) On Tue, Feb 15, 2011 at 10:25 AM, holger krekel wrote: > Hi all, > > what do you think or know why Google choose Java over Python for > its Android system? ?Surely, Python doesn't have a big track record > in terms of targetting mobile devices. ?Also CPython is limited > in how it can change and adapt i guess. ?I imagine PyPy could do much > better. ?Wouldn't it be cool to have a totally python-based > state-of-the-art phone? :) > > cheers, > holger > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From pypy at pocketnix.org Tue Feb 15 15:29:54 2011 From: pypy at pocketnix.org (pypy at pocketnix.org) Date: Tue, 15 Feb 2011 14:29:54 +0000 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <4D5A884F.9020405@gmx.de> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <201102151151.p1FBpnsk024534@theraft.openend.se> <4D5A884F.9020405@gmx.de> Message-ID: <20110215142954.GB10549@pocketnix.org> On Tue, Feb 15, 2011 at 03:06:07PM +0100, Carl Friedrich Bolz wrote: > On 02/15/2011 12:51 PM, Laura Creighton wrote: > > http://www.freedominput.com/freedom-accessories/freedom-pro-keyboard which > > works with Android. > > > > which makes the whole idea of being able to sit down and program whenever > > the mood strikes you, even when your laptop is far away or out of power > > more attractive, since you can touchtype. > > > > But that doesn't make me want to start programming in java. > > Especially for the sort of quick and dirty calculations I use python > > -c for. > > I am sure you know of this then: > > http://code.google.com/p/android-scripting/ > > It's an app for Android that lets you script the phone with various > dynamic languages (including Python). Actually I think that PyPy > could/should be made to work with this thing. It would not quite make > PyPy a first class VM on the phone, but would already give you access to > a number of phone-specific features. the Scripting eniroment uses an enviroment varible to indicate a socket to connect to, once connected you make JSON calls over the socket to access the platform API's that are done via a java proxy. porting pypy to run in the enviroment would be as simple as compiling an arm port and importing the (all python) proxy libary i have a couple of scripts running on my phone for utility functions (checking my site is up and such) the API is a bit limited and i think that pypy running on the native VM would be more flexible for creating "full" programs (cant acceses many GUI componenets from this restricted enviroment) From dimaqq at gmail.com Tue Feb 15 18:22:04 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 15 Feb 2011 10:22:04 -0700 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <20110215102506.GM30557@trillke.net> References: <20110215102506.GM30557@trillke.net> Message-ID: The real answers: 1. Google bought Android (company) 2. Java allows execution of untrusted 3rd party code 3, maybe, Java was "mature" around the time of acquisition, python 2.4 was released and there was psyco, but that's x86 only, when Android Inc started I guess they didn't have a serious python option at all. tools like IDEs and debuggers and hoards of underpaid Java developers in the wild surely contributed too. This topic is really not for pypy. My 2c. On 15 February 2011 03:25, holger krekel wrote: > Hi all, > > what do you think or know why Google choose Java over Python for > its Android system? ?Surely, Python doesn't have a big track record > in terms of targetting mobile devices. ?Also CPython is limited > in how it can change and adapt i guess. ?I imagine PyPy could do much > better. ?Wouldn't it be cool to have a totally python-based > state-of-the-art phone? :) > > cheers, > holger > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From drsalists at gmail.com Tue Feb 15 19:40:37 2011 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 15 Feb 2011 10:40:37 -0800 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: <20110215142954.GB10549@pocketnix.org> References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <201102151151.p1FBpnsk024534@theraft.openend.se> <4D5A884F.9020405@gmx.de> <20110215142954.GB10549@pocketnix.org> Message-ID: On Tue, Feb 15, 2011 at 6:29 AM, wrote: > On Tue, Feb 15, 2011 at 03:06:07PM +0100, Carl Friedrich Bolz wrote: >> On 02/15/2011 12:51 PM, Laura Creighton wrote: > >> I am sure you know of this then: >> >> http://code.google.com/p/android-scripting/ >> >> It's an app for Android that lets you script the phone with various >> dynamic languages (including Python). Actually I think that PyPy >> could/should be made to work with this thing. It would not quite make >> PyPy a first class VM on the phone, but would already give you access to >> a number of phone-specific features. > > the Scripting eniroment uses an enviroment varible to indicate a > socket to connect to, once connected you make JSON calls over the > socket to access the platform API's that are done via a java proxy. > porting pypy to run in the enviroment would be as simple as compiling > an arm port and importing the (all python) proxy libary A guy at my local Python User Group uses an Android phone, and has explored both Python and Ruby in the Android Scripting Environment. He tells me that because the Ruby implementation is running on the Dalvik virtual machine, that its API access is more complete than that of the CPython port. Presumably you need a wrapper function for everything you want to chain into over the socket (in CPython), and people create those wrappers mostly on an as-needed basis. In light of this, it seems it might be better to put a JVM (which is almost the same as Dalvik) version of Pypy, or Jython, on Android rather than Pypy built for ARM. From dimaqq at gmail.com Tue Feb 15 20:09:49 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 15 Feb 2011 12:09:49 -0700 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <201102151151.p1FBpnsk024534@theraft.openend.se> <4D5A884F.9020405@gmx.de> <20110215142954.GB10549@pocketnix.org> Message-ID: There's always NDK which includes JNI bindings, so you can comile (or jit-compile) pypy and python code natively and still access android framework. If it really is a good idea, I cannot tell. I suppose if you want simple scripting, json-rpc would suit you fine, if you wanted more scripting, reflection, shared object space and good integration perhaps dalvik jvm is a way to go, if cpython or pypy becomes standard in android and apps are witten in python entirely, jni bindings are probably better, whether static or runtime reflection. I'm sure there would be a few issues to resolve though, object lifetime, garbage collection and callbacks for one. 2c from someone who went dumbphone 2 years ago :P On 15 February 2011 11:40, Dan Stromberg wrote: > On Tue, Feb 15, 2011 at 6:29 AM, ? wrote: >> On Tue, Feb 15, 2011 at 03:06:07PM +0100, Carl Friedrich Bolz wrote: >>> On 02/15/2011 12:51 PM, Laura Creighton wrote: >> > >>> I am sure you know of this then: >>> >>> http://code.google.com/p/android-scripting/ >>> >>> It's an app for Android that lets you script the phone with various >>> dynamic languages (including Python). Actually I think that PyPy >>> could/should be made to work with this thing. It would not quite make >>> PyPy a first class VM on the phone, but would already give you access to >>> a number of phone-specific features. >> >> the Scripting eniroment uses an enviroment varible to indicate a >> socket to connect to, once connected you make JSON calls over the >> socket to access the platform API's that are done via a java proxy. >> porting pypy to run in the enviroment would be as simple as compiling >> an arm port and importing the (all python) proxy libary > > A guy at my local Python User Group uses an Android phone, and has > explored both Python and Ruby in the Android Scripting Environment. > > He tells me that because the Ruby implementation is running on the > Dalvik virtual machine, that its API access is more complete than that > of the CPython port. ?Presumably you need a wrapper function for > everything you want to chain into over the socket (in CPython), and > people create those wrappers mostly on an as-needed basis. > > In light of this, it seems it might be better to put a JVM (which is > almost the same as Dalvik) version of Pypy, or Jython, on Android > rather than Pypy built for ARM. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From stefan_ml at behnel.de Wed Feb 16 07:32:36 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 16 Feb 2011 07:32:36 +0100 Subject: [pypy-dev] google should build a pypy phone or someone In-Reply-To: References: <20110215102506.GM30557@trillke.net> <4D5A573B.40403@gmail.com> <201102151151.p1FBpnsk024534@theraft.openend.se> <4D5A884F.9020405@gmx.de> <20110215142954.GB10549@pocketnix.org> Message-ID: Dima Tisnek, 15.02.2011 20:09: > 2c from someone who went dumbphone 2 years ago :P Yay to phones that allow you to call other people without trying to distract you from what you were just thinking to do. Stefan From bea at changemaker.nu Wed Feb 16 12:01:38 2011 From: bea at changemaker.nu (Bea During) Date: Wed, 16 Feb 2011 12:01:38 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: <4D5A887A.2040206@gmail.com> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> <4D5A8774.3050700@gmx.de> <4D5A887A.2040206@gmail.com> Message-ID: <4D5BAE92.2080306@changemaker.nu> Hi there Regarding PyPy sprints in Europe during 2011: Antonio Cuni skrev: > On 15/02/11 15:02, Carl Friedrich Bolz wrote: > > >> Also I guess it is likely that we will have another one in D?sseldorf >> later in the year. >> > > and probably one after europython, as usual? > > ciao, > Anto > Here is a suggestion of places and dates based on Lauras, Carl Friedrich and Antos input: - Gothenburg sprint: 25th of April to 1st of May - Europython sprint/Florence: 25th of June to 26th of June (EP2011 official sprint dates) - D?sseldorf sprint: 22nd of August to 28th of August (fits the plan of the funded PyJIT project which ends end August) What do you think about these dates - would they work? Cheers Bea From lac at openend.se Wed Feb 16 13:08:07 2011 From: lac at openend.se (Laura Creighton) Date: Wed, 16 Feb 2011 13:08:07 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: Message from Bea During of "Wed, 16 Feb 2011 12:01:38 +0100." <4D5BAE92.2080306@changemaker.nu> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> <4D5A8774.3050700@gmx.de> <4D5A887A.2040206@gmail.com> <4D5BAE92.2080306@changemaker.nu> Message-ID: <201102161208.p1GC875O027115@theraft.openend.se> In a message of Wed, 16 Feb 2011 12:01:38 +0100, Bea During writes: >Here is a suggestion of places and dates based on Lauras, Carl Friedrich >and Antos >input: > >- Gothenburg sprint: 25th of April to 1st of May >- Europython sprint/Florence: 25th of June to 26th of June (EP2011 >official sprint dates) >- D?sseldorf sprint: 22nd of August to 28th of August (fits the plan of >the funded PyJIT project which ends end August) > >What do you think about these dates - would they work? > >Cheers > >Bea For me: The G?teborg dates work. The Florence dates also work, but we might want to consider staying around longer, because 2 days of sprint is awfully short. And I have no way to know if any dates in July or August will work until the members of N?set's Paddlarklub decide which weeks will be spend on sea kayaking tours. I don't know when that will be decided -- maybe Jacob does -- but not in the next week at any rate. Laura From william.leslie.ttg at gmail.com Wed Feb 16 13:24:01 2011 From: william.leslie.ttg at gmail.com (William ML Leslie) Date: Wed, 16 Feb 2011 23:24:01 +1100 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: Message-ID: On 15 February 2011 12:13, Maciej Fijalkowski wrote: > Hi. > > There is growing interest about PyPy and especially about extension > modules. Apparently there are some people (like Alex) that are willing > to write modules in RPython that should not go to the main tree. Since > separate compilation is considered hard, how hard would it be to > provide separate loading? This would mean you still compile the whole > interpreter, but you can load the module from a compiled PyPy that > didn't have this option on. 0. What do you do about linking? Are rpython class and function names mangled consistently enough that a module compiled against one patch-level runtime with one set of options will link against another patch version with some different options? 1. Is it reasonable to ensure that *all* symbols that may be visible to an extension module are exported by the runtime? f may be inlined into g, but if f may still be callable by an extension module, it must be available by being exported static, too. 2. I always assumed this ("separate loading", ie, common translation) was the intended way to do separate compilation in rpython *anyway*; but there is one nagging thing. In order to define the boundary of the compilation, you already need to declare the interface of the module. Maybe not the annotations of the arguments, but at least which functions, when the annotator sees them, should be placed in your ".so". For the specific case of modules in the pypy python runtime this can be the interpleveldefs attribute of Module objects, but such functions all seem to have known signature annotation; they take a space and a number of wrapped arguments and return a wrapped object. Armed with this and the signature annotations of the functions in the core runtime, it is reasonable to expect that we can determine which functions belong in the ".so". Exactly how those core annotations are obtained doesn't need to be set in stone - obtained within the same translation or annotation performed earlier are both reasonable and not incompatible places to start. In both cases, the annotation is *computed* in the usual way. The alternative is defining the interface exported by the runtime explicitly, which is a mammoth task, and I don't think anyone has suggested it: is that what you are arguing against, fijal? There are still some details, such as how you export JitCodes from functions in your module, and what it means to do so - but nothing prohibitive that I can see. On 15 February 2011 19:32, Paolo Giarrusso wrote: > On Tue, Feb 15, 2011 at 09:17, Dima Tisnek wrote: >> I assume here that modules don't introduce dependencies into the iterpreter. >> I guess in the long run this ought to be the case, right? > > I don't think you can guarantee this. Type inference is global, and > you might need a user for each API to better infer its type. Maybe > uses of an API in testcases allow fully inferring their types, but I'd > guess not. If this does happen, it makes the callee part of a public interface, which should probably be explicitly annotated. I think checking for this case (where annotation in the extension would generalise the type signature of a dependency) is not too difficult. > However, what is true in general is that if less specific types are > inferred, that affects just performance, not correctness (I don't know > if that's true of PyPy, but you ought to be able to pass "object"s > around). Maybe the slowdown is insignificant, maybe it is a huge > problem, maybe few annotations can save the day. Annotation widens, rather than narrows: overly specific types are more likely to be inferred. > However, it is still not clear (to me) where previous efforts stopped. > Is it hard to: > 1) devise an algorithm like Dima proposed > or to > 2) implement it (because of too much code to change and limited manpower) > or to > 3) or to have a small performance loss? I understood that there was a lot of design to be done and there were other priorities. Devising an algorithm is not difficult, specifying it in terms of our existing annotation & flow model is slightly more so. > Per-file separate compilation would likely fall into 3), because too > little type inference would happen, isn't it? I suspect you are thinking of accidentally boxing interp-level integers or something, but that is an impossible condition (the dreaded SomeObject annotation). If not, where do you think a loss would come from? -- William Leslie From fuzzyman at voidspace.org.uk Wed Feb 16 13:30:22 2011 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Feb 2011 12:30:22 +0000 Subject: [pypy-dev] Separate compilation and friends In-Reply-To: References: Message-ID: On 15 February 2011 01:13, Maciej Fijalkowski wrote: > Hi. > > There is growing interest about PyPy and especially about extension > modules. Apparently there are some people (like Alex) that are willing > to write modules in RPython that should not go to the main tree. As a side note I think Alex's code *should* go in the main tree. It is really *needed* to work with django and pypy (at least for those using postgres which is a significant proportion of django users) and it would be much better [for pypy] if that "just worked" out of the box. Michael > Since > separate compilation is considered hard, how hard would it be to > provide separate loading? This would mean you still compile the whole > interpreter, but you can load the module from a compiled PyPy that > didn't have this option on. > > Cheers, > fijal > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Wed Feb 16 13:33:12 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 16 Feb 2011 13:33:12 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: <4D5BAE92.2080306@changemaker.nu> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> <4D5A8774.3050700@gmx.de> <4D5A887A.2040206@gmail.com> <4D5BAE92.2080306@changemaker.nu> Message-ID: <4D5BC408.4060900@gmail.com> On 16/02/11 12:01, Bea During wrote: > Here is a suggestion of places and dates based on Lauras, Carl Friedrich and > Antos > input: > > - Gothenburg sprint: 25th of April to 1st of May > - Europython sprint/Florence: 25th of June to 26th of June (EP2011 official > sprint dates) > - D?sseldorf sprint: 22nd of August to 28th of August (fits the plan of the > funded PyJIT project which ends end August) > > What do you think about these dates - would they work? for me they "mostly" work, although I might arrive one day later or depart one day earlier for the Gothenburg and Duesseldorf ones. ciao, Anto From bea at changemaker.nu Wed Feb 16 14:31:17 2011 From: bea at changemaker.nu (Bea During) Date: Wed, 16 Feb 2011 14:31:17 +0100 Subject: [pypy-dev] European sprints? In-Reply-To: <4D5BC408.4060900@gmail.com> References: <58722.170.148.215.157.1297760241.squirrel@mail5.webfaction.com> <201102151054.p1FAsZWb018955@theraft.openend.se> <4D5A8774.3050700@gmx.de> <4D5A887A.2040206@gmail.com> <4D5BAE92.2080306@changemaker.nu> <4D5BC408.4060900@gmail.com> Message-ID: <4D5BD1A5.4050102@changemaker.nu> Hi there Antonio Cuni skrev: > On 16/02/11 12:01, Bea During wrote: > > >> Here is a suggestion of places and dates based on Lauras, Carl Friedrich and >> Antos >> input: >> >> - Gothenburg sprint: 25th of April to 1st of May >> - Europython sprint/Florence: 25th of June to 26th of June (EP2011 official >> sprint dates) >> - D?sseldorf sprint: 22nd of August to 28th of August (fits the plan of the >> funded PyJIT project which ends end August) >> >> What do you think about these dates - would they work? >> > > for me they "mostly" work, although I might arrive one day later or depart one > day earlier for the Gothenburg and Duesseldorf ones. > > ciao, > Anto > > My main idea with these suggested dates is to be able to state that these are our known sprint plans so far on the PyPy blog. We need to detail them more when we get closer and announce them through the usual channels. So if this would fit for D?sseldorf we should post on the blog about it. I can also see the need to sprint longer than 2 days at EP 2011 but we need to talk to the organizers to see if they have facilities available (this has been an issue during previous conferences). Cheers Bea From holger at merlinux.eu Thu Feb 17 15:23:06 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 17 Feb 2011 15:23:06 +0100 Subject: [pypy-dev] further pypy repo migrations Message-ID: <20110217142305.GC30557@trillke.net> hi all, i'd like to continue consolidation of PyPy repositories (and more generally pypy services but that in a separate mail). Thanks to Ronny Pfannschmidt, sponsorship from merlinux (my company), and work from Armin, Anto and others the main PyPy development repository has been migrated to mercurial on Bitbucket. Btw, we just asked and Bitbucket nicely upgraded the PyPy project account to become an unlimited one, thanks! Now we still have some more repositories on codespeak svn. One is "extradoc" which contains a lot of talk, sprint, misc infos and the "pypy.org" website content. We can either move this wholesale to an "extradoc" one on bitbucket or split it maybe into "pypy.org" and "extradoc". Note that the old codespeak.net subversion URLs will remain valid for quite some time so any cleanup/removal of dirs should better happen in the new mercurial repo. Another repo is "pypy-z", maybe not known to most. It is a repository where core project members keep information about more internal bits, like our previous contracts with Google, the European Union and some un-named entities (ask your favourite PSU representative for more info). I suggest we move this to become a "private" mercurial repo on bitbucket. Any more pypy related repositories that need migration or any notes/comments? best holger -- From anto.cuni at gmail.com Thu Feb 17 15:48:04 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 17 Feb 2011 15:48:04 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <20110217142305.GC30557@trillke.net> References: <20110217142305.GC30557@trillke.net> Message-ID: <4D5D3524.3040509@gmail.com> On 17/02/11 15:23, holger krekel wrote: > Thanks to Ronny Pfannschmidt, sponsorship from merlinux (my company), and work > from Armin, Anto and others the main PyPy development repository has been > migrated to mercurial on Bitbucket. Btw, we just asked and Bitbucket nicely > upgraded the PyPy project account to become an unlimited one, thanks! just to clarify: "unlimited account" means that we can have private repositories with as many users as we want (which is what we needed for pypy-z). Thanks to Bitbucket also from my side :-) > Any more pypy related repositories that need migration > or any notes/comments? currently, pypy has three svn subrepos which live on codespeak: greenlet = [svn]http://codespeak.net/svn/greenlet/trunk/c testrunner = [svn]http://codespeak.net/svn/pypy/build/testrunner lib_pypy/pyrepl = [svn]http://codespeak.net/svn/pyrepl/trunk/pyrepl/pyrepl we will need to migrate those as well at some point, and probably make them mercurial subrepositories instead of svn ones. ciao, Anto From lac at openend.se Thu Feb 17 17:43:20 2011 From: lac at openend.se (Laura Creighton) Date: Thu, 17 Feb 2011 17:43:20 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: Message from holger krekel of "Thu, 17 Feb 2011 15:23:06 +0100." <20110217142305.GC30557@trillke.net> References: <20110217142305.GC30557@trillke.net> Message-ID: <201102171643.p1HGhKDX019756@theraft.openend.se> In a message of Thu, 17 Feb 2011 15:23:06 +0100, holger krekel writes: >Any more pypy related repositories that need migration >or any notes/comments? > >best >holger I just found a surprising number (at least it surprised me) of references in my own stuff to /svn/user/arigo/hack/ Laura From tismer at stackless.com Thu Feb 17 18:15:57 2011 From: tismer at stackless.com (Christian Tismer) Date: Thu, 17 Feb 2011 18:15:57 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D5D3524.3040509@gmail.com> References: <20110217142305.GC30557@trillke.net> <4D5D3524.3040509@gmail.com> Message-ID: <4D5D57CD.90100@stackless.com> On 2/17/11 3:48 PM, Antonio Cuni wrote: > On 17/02/11 15:23, holger krekel wrote: > >> Thanks to Ronny Pfannschmidt, sponsorship from merlinux (my company), and work >> from Armin, Anto and others the main PyPy development repository has been >> migrated to mercurial on Bitbucket. Btw, we just asked and Bitbucket nicely >> upgraded the PyPy project account to become an unlimited one, thanks! > just to clarify: "unlimited account" means that we can have private > repositories with as many users as we want (which is what we needed for > pypy-z). Thanks to Bitbucket also from my side :-) > > >> Any more pypy related repositories that need migration >> or any notes/comments? > currently, pypy has three svn subrepos which live on codespeak: > > greenlet = [svn]http://codespeak.net/svn/greenlet/trunk/c > testrunner = [svn]http://codespeak.net/svn/pypy/build/testrunner > lib_pypy/pyrepl = [svn]http://codespeak.net/svn/pyrepl/trunk/pyrepl/pyrepl > > we will need to migrate those as well at some point, and probably make them > mercurial subrepositories instead of svn ones. > > ciao, > Anto > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev Hi folks, if it helps, then I can offer to build a repos or website on my machine. I have a big machine with plenty of resources, which are only little used. Maybe for something that doesn't fit bitbucket very well, or as a mirror site, whatever makes sense. Right now I'm hosting starship.python.net and stackless.com as virtual machines, plus some experimental hidden servers. I'd be happy to donate an OpenVZ Debian server of reasonable size. Just let me know. cheers - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 173 24 18 776 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From arigo at tunes.org Fri Feb 18 16:18:15 2011 From: arigo at tunes.org (Armin Rigo) Date: Fri, 18 Feb 2011 16:18:15 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <201102171643.p1HGhKDX019756@theraft.openend.se> References: <20110217142305.GC30557@trillke.net> <201102171643.p1HGhKDX019756@theraft.openend.se> Message-ID: Hi Laura, On Thu, Feb 17, 2011 at 5:43 PM, Laura Creighton wrote: > I just found a surprising number (at least it surprised me) of references > in my own stuff to /svn/user/arigo/hack/ I moved this part of the repository to http://bitbucket.org/arigo/arigo . Armin From cfbolz at gmx.de Fri Feb 18 12:14:47 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 18 Feb 2011 12:14:47 +0100 Subject: [pypy-dev] CfP: ICOOOLPS 2011 Message-ID: <4D5E54A7.5000908@gmx.de> Hi all, I think we should consider sending a paper to ICOOOLPS this year again, see call for papers below. Carl Friedrich -- Call For Papers for the sixth annual workshop on Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems (ICOOOLPS) to be held at ECOOP 2011 in Lancaster, UK on July 26th 2011 http://www.icooolps.info/ Overview: Computer programming languages, especially Object-Oriented languages, are pervasive and play a significant role in computer science and engineering life and sometime appear as ubiquitous and completely mature. However, despite a large number of works, there still is a clear need for solutions for efficient implementation and compilation of OO languages in various application domains ranging from embedded and real-time systems to desktop systems. The ICOOOLPS workshop thus aims to address this crucial issue of optimization, mainly but not only in OO languages, programs and systems. It intends to do so by bringing together researchers and practitioners working in the field of implementation and optimization of programming languages. Its main goals are identifying fundamental bases and key current issues pertaining to the efficient implementation, compilation and optimization of OO languages, and outlining future challenges and research directions. An expected output of this workshop is a synthesis identifying fundamental bases and key current issues pertaining to the efficient implementation and compilation of OO languages, in order to spread them further amongst the various computing systems. It is also intended to extend this synthesis to encompass future challenges and research directions in the field of OO languages implementation and optimization, as well as non-OO languages. Important Dates: Workshop paper submission: April 15, 2011 Workshop paper acceptance: May 20, 2011 Workshop: July 26, 2011 Program Committee: Mark van den Brand TU Eindhoven, The Netherlands Peter Dickman Google, Switzerland Roland Ducournau LIRMM, France M. Anton Ertl Technical University of Vienna (TU Wien), Austria Remi Forax University of Marne-la-Vall?e, France Bj?rn Franke University of Edinburgh, UK Andreas Gal Mozilla Corporation, USA Kevin Hammond University of St Andrews, UK Tim Harris Microsoft Research, UK Michael Haupt Hasso-Plattner-Institut, Germany Eric Jul (co-chair) DIKU, The University of Copenhagen, Denmark Tomas Kalibera The University of Kent, UK Stein Krogdahl University of Oslo, Norway Francis Chi Moon Lau The University of Hong Kong, Hong Kong Ian Rogers (chair) Azul Systems, USA Jennifer Sartor Ecole Polytechnique Federale de Lausanne, Switzerland Manuel Serrano INRIA, France Mario Wolczko Oracle Labs, USA Olivier Zendra (co-chair) INRIA/LORIA, France From alex.gaynor at gmail.com Fri Feb 18 17:55:27 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Fri, 18 Feb 2011 11:55:27 -0500 Subject: [pypy-dev] CfP: ICOOOLPS 2011 In-Reply-To: <4D5E54A7.5000908@gmx.de> References: <4D5E54A7.5000908@gmx.de> Message-ID: On Fri, Feb 18, 2011 at 6:14 AM, Carl Friedrich Bolz wrote: > Hi all, > > I think we should consider sending a paper to ICOOOLPS this year again, > see call for papers below. > > Carl Friedrich > > > -- > > Call For Papers for the sixth annual workshop on Implementation, > Compilation, Optimization of Object-Oriented Languages, Programs and > Systems (ICOOOLPS) to be held at ECOOP 2011 in Lancaster, UK on July > 26th 2011 > > http://www.icooolps.info/ > > Overview: > > Computer programming languages, especially Object-Oriented languages, > are pervasive and play a significant role in computer science and > engineering life and sometime appear as ubiquitous and completely > mature. However, despite a large number of works, there still is a clear > need for solutions for efficient implementation and compilation of OO > languages in various application domains ranging from embedded and > real-time systems to desktop systems. > > The ICOOOLPS workshop thus aims to address this crucial issue of > optimization, mainly but not only in OO languages, programs and systems. > It intends to do so by bringing together researchers and practitioners > working in the field of implementation and optimization of programming > languages. Its main goals are identifying fundamental bases and key > current issues pertaining to the efficient implementation, compilation > and optimization of OO languages, and outlining future challenges and > research directions. > > An expected output of this workshop is a synthesis identifying > fundamental bases and key current issues pertaining to the efficient > implementation and compilation of OO languages, in order to spread them > further amongst the various computing systems. It is also intended to > extend this synthesis to encompass future challenges and research > directions in the field of OO languages implementation and optimization, > as well as non-OO languages. > > Important Dates: > > Workshop paper submission: April 15, 2011 > Workshop paper acceptance: May 20, 2011 > Workshop: July 26, 2011 > > Program Committee: > > Mark van den Brand TU Eindhoven, The Netherlands > Peter Dickman Google, Switzerland > Roland Ducournau LIRMM, France > M. Anton Ertl Technical University of Vienna (TU Wien), Austria > Remi Forax University of Marne-la-Vall?e, France > Bj?rn Franke University of Edinburgh, UK > Andreas Gal Mozilla Corporation, USA > Kevin Hammond University of St Andrews, UK > Tim Harris Microsoft Research, UK > Michael Haupt Hasso-Plattner-Institut, Germany > Eric Jul (co-chair) DIKU, The University of Copenhagen, Denmark > Tomas Kalibera The University of Kent, UK > Stein Krogdahl University of Oslo, Norway > Francis Chi Moon Lau The University of Hong Kong, Hong Kong > Ian Rogers (chair) Azul Systems, USA > Jennifer Sartor Ecole Polytechnique Federale de Lausanne, > Switzerland > Manuel Serrano INRIA, France > Mario Wolczko Oracle Labs, USA > Olivier Zendra (co-chair) INRIA/LORIA, France > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev Do you have any topics you think we should consider submitting? Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From cfbolz at gmx.de Mon Feb 21 18:12:22 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 21 Feb 2011 18:12:22 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <20110217142305.GC30557@trillke.net> References: <20110217142305.GC30557@trillke.net> Message-ID: <4D629CF6.4010007@gmx.de> On 02/17/2011 03:23 PM, holger krekel wrote: > i'd like to continue consolidation of PyPy repositories (and more generally > pypy services but that in a separate mail). > > Thanks to Ronny Pfannschmidt, sponsorship from merlinux (my company), and work > from Armin, Anto and others the main PyPy development repository has been > migrated to mercurial on Bitbucket. Btw, we just asked and Bitbucket nicely > upgraded the PyPy project account to become an unlimited one, thanks! > > Now we still have some more repositories on codespeak svn. One is "extradoc" > which contains a lot of talk, sprint, misc infos and the "pypy.org" website > content. We can either move this wholesale to an "extradoc" one on bitbucket or > split it maybe into "pypy.org" and "extradoc". Note that the old codespeak.net > subversion URLs will remain valid for quite some time so any cleanup/removal of > dirs should better happen in the new mercurial repo. > > Another repo is "pypy-z", maybe not known to most. It is a repository where > core project members keep information about more internal bits, like our > previous contracts with Google, the European Union and some un-named entities > (ask your favourite PSU representative for more info). I suggest we move this > to become a "private" mercurial repo on bitbucket. > > Any more pypy related repositories that need migration > or any notes/comments? There are also the repos in svn/pypy/lang that would need conversion. At least the Smalltalk one has branches in its history, maybe it's not important to keep the precise branch history though. Another consideration I have is that we have been handing out external pointers into our repositories from various places, e.g. from our blog posts and also in some papers. I don't see an easy solution to keep them valid, which is a bit of a pity. Cheers, Carl Friedrich From dcolish at gmail.com Tue Feb 22 03:39:47 2011 From: dcolish at gmail.com (Dan Colish) Date: Mon, 21 Feb 2011 18:39:47 -0800 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D629CF6.4010007@gmx.de> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> Message-ID: <3368745349BF49BD9866112698CBAF3B@gmail.com> On Monday, February 21, 2011 at 9:12 AM, Carl Friedrich Bolz wrote: On 02/17/2011 03:23 PM, holger krekel wrote: > > i'd like to continue consolidation of PyPy repositories (and more generally > > pypy services but that in a separate mail). > > > > Thanks to Ronny Pfannschmidt, sponsorship from merlinux (my company), and work > > from Armin, Anto and others the main PyPy development repository has been > > migrated to mercurial on Bitbucket. Btw, we just asked and Bitbucket nicely > > upgraded the PyPy project account to become an unlimited one, thanks! > > > > Now we still have some more repositories on codespeak svn. One is "extradoc" > > which contains a lot of talk, sprint, misc infos and the "pypy.org" website > > content. We can either move this wholesale to an "extradoc" one on bitbucket or > > split it maybe into "pypy.org" and "extradoc". Note that the old codespeak.net > > subversion URLs will remain valid for quite some time so any cleanup/removal of > > dirs should better happen in the new mercurial repo. > > > > Another repo is "pypy-z", maybe not known to most. It is a repository where > > core project members keep information about more internal bits, like our > > previous contracts with Google, the European Union and some un-named entities > > (ask your favourite PSU representative for more info). I suggest we move this > > to become a "private" mercurial repo on bitbucket. > > > > Any more pypy related repositories that need migration > > or any notes/comments? > > There are also the repos in svn/pypy/lang that would need conversion. At > least the Smalltalk one has branches in its history, maybe it's not > important to keep the precise branch history though. > > Another consideration I have is that we have been handing out external > pointers into our repositories from various places, e.g. from our blog > posts and also in some papers. I don't see an easy solution to keep them > valid, which is a bit of a pity. I'd be happy to write a forwarding application that can redirect any requests to the codespeak svn. This would allow the group to move these additional repositories to bitbucket, but not break existing links. With a good url mapping, this application could serve to forward most codespeak projects beyond pypy, which might be enticing for the codespeak admins. The application is fairly simple, so let me know if there is interest and I'll finish up the preliminary code that I've got done now. --Dan From dcolish at gmail.com Tue Feb 22 04:52:10 2011 From: dcolish at gmail.com (Dan Colish) Date: Mon, 21 Feb 2011 19:52:10 -0800 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D629CF6.4010007@gmx.de> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> Message-ID: Just a quick update, I've sketched out a working bouncer for just the pypy svn trunk. Its living here for now: https://gist.github.com/838189 -- Dan From dcolish at gmail.com Tue Feb 22 04:57:33 2011 From: dcolish at gmail.com (Dan Colish) Date: Mon, 21 Feb 2011 19:57:33 -0800 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D629CF6.4010007@gmx.de> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> Message-ID: <69510A9460C44204AC8F89603528560C@gmail.com> Errr, sorry I lied. The code's here https://bitbucket.org/dcolish/pypy-bouncer. There were a few more files that made it worth getting into a proper vcs repo. -- Dan From luke at freeculture.co.uk Tue Feb 22 15:11:22 2011 From: luke at freeculture.co.uk (Luke Taylor) Date: Tue, 22 Feb 2011 14:11:22 +0000 Subject: [pypy-dev] Couple of questions... Message-ID: Hi everybody, I'm not a developer but I'm a keen fan of pypy and I have been watching the project for about 6 months now. I'm interested to know.. if there are any plans to make PyPy implement Python 3.x ? Why don't extension modules for Python with bits written in C (eg. SciPy) work with PyPy? Thanks for your time, Luke -------------- next part -------------- An HTML attachment was scrubbed... URL: From dimaqq at gmail.com Tue Feb 22 18:40:01 2011 From: dimaqq at gmail.com (Dima Tisnek) Date: Tue, 22 Feb 2011 10:40:01 -0700 Subject: [pypy-dev] Couple of questions... In-Reply-To: References: Message-ID: IIUC 2.7 support branch was being merged recently but not finished (?) and that's the natural way towards 3.x On 22 February 2011 07:11, Luke Taylor wrote: > Hi everybody, > > I'm not a developer but I'm a keen fan of pypy and I have been watching the > project for about 6 months now. > > I'm interested to know.. > > if there are any plans to make PyPy implement Python 3.x ? > > Why don't extension modules for Python with bits written in C (eg. SciPy) > work with PyPy? > > > Thanks for your time, > > > Luke > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From cfbolz at gmx.de Tue Feb 22 18:48:08 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 22 Feb 2011 18:48:08 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <3368745349BF49BD9866112698CBAF3B@gmail.com> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> <3368745349BF49BD9866112698CBAF3B@gmail.com> Message-ID: <4D63F6D8.2010702@gmx.de> Hi Dan, On 02/22/2011 03:39 AM, Dan Colish wrote: > I'd be happy to write a forwarding application that can redirect any > requests to the codespeak svn. This would allow the group to move > these additional repositories to bitbucket, but not break existing > links. With a good url mapping, this application could serve to > forward most codespeak projects beyond pypy, which might be enticing > for the codespeak admins. The application is fairly simple, so let me > know if there is interest and I'll finish up the preliminary code > that I've got done now. That sounds very cool. I have no clue how that would be integrated in whatever codespeak is going to end up being. Carl Friedrich From dcolish at gmail.com Tue Feb 22 19:21:00 2011 From: dcolish at gmail.com (Dan Colish) Date: Tue, 22 Feb 2011 10:21:00 -0800 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D63F6D8.2010702@gmx.de> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> <3368745349BF49BD9866112698CBAF3B@gmail.com> <4D63F6D8.2010702@gmx.de> Message-ID: <68C0B5EF6E354467AD4DE1B2A75E13FE@gmail.com> On Tuesday, February 22, 2011 at 9:48 AM, Carl Friedrich Bolz wrote: Hi Dan, > > On 02/22/2011 03:39 AM, Dan Colish wrote: > > I'd be happy to write a forwarding application that can redirect any > > requests to the codespeak svn. This would allow the group to move > > these additional repositories to bitbucket, but not break existing > > links. With a good url mapping, this application could serve to > > forward most codespeak projects beyond pypy, which might be enticing > > for the codespeak admins. The application is fairly simple, so let me > > know if there is interest and I'll finish up the preliminary code > > that I've got done now. > > That sounds very cool. I have no clue how that would be integrated in > whatever codespeak is going to end up being. > Who can I contact at codespeak to begin a discussion on integrating this bouncer application? I've got a working draft that seems to do the trick up on bitbucket now, so I think I can grind this bit of the migration out pretty quickly. --Dan From lac at openend.se Tue Feb 22 20:00:30 2011 From: lac at openend.se (Laura Creighton) Date: Tue, 22 Feb 2011 20:00:30 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: Message from Dan Colish of "Tue, 22 Feb 2011 10:21:00 PST." <68C0B5EF6E354467AD4DE1B2A75E13FE@gmail.com> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> <3368745349BF49BD9866112698CBAF3B@gmail.com> <4D63F6D8.2010702@gmx.de> <68C0B5EF6E354467AD4DE1B2A75E13FE@gmail.com> Message-ID: <201102221900.p1MJ0UG0030841@theraft.openend.se> Holger Krekel is the person to talk with. But he hasn't been around for a few days. I think he is on vacation. Laura From arigo at tunes.org Wed Feb 23 16:03:02 2011 From: arigo at tunes.org (Armin Rigo) Date: Wed, 23 Feb 2011 16:03:02 +0100 Subject: [pypy-dev] Couple of questions... In-Reply-To: References: Message-ID: Hi Luke, On Tue, Feb 22, 2011 at 3:11 PM, Luke Taylor wrote: > if there are any plans to make PyPy implement Python 3.x ? There are no plans at the moment, though I imagine that such plans will emerge at one point. Indeed, we are almost done with supporting 2.7, which is "on the way there". > Why don't extension modules for Python with bits written in C (eg. SciPy) > work with PyPy? You can't generalize like that; some of them do, thanks to the cpyext module of PyPy. But not SciPy, which is using a lot of fonctionality of CPython that is "semi-internal". Moreover, cpyext is slow, so it's rather meant to support C extensions like wxWindow that are not there for performance but for functionality. We can improve performance a bit, but it's unlikely to ever get at a level where it's appropriate for NumPy or SciPy. Instead, what needs to be done there is rewrite some parts in a more PyPy-friendly way. (For NumPy there is a project at http://projects.scipy.org/numpy/wiki/NumPyRefactoring that is likely to be useful for us too.) A bient?t, Armin. From dcolish at gmail.com Wed Feb 23 17:29:53 2011 From: dcolish at gmail.com (Dan Colish) Date: Wed, 23 Feb 2011 08:29:53 -0800 Subject: [pypy-dev] Writing Mixed Modules for MacOS Message-ID: I've been working on porting the OSX specific libraries in CPython to PyPy and I'm running into issues. The shape of many fundamental data structures used in the CoreFoundation libraries are not available to me. I was hoping to just fudge it by using a ptr to that structure, but it is not working out as I had hoped. Here is a terribly incorrect sample of what I've been hacking on: http://paste.pocoo.org/show/yENK4gE11yRM9i6p46Ra and here is the corresponding osx documentation: http://goo.gl/i6mYD. I was looking at the approach taken for windows, but it is not clear to me where the fundamental structures for windows lie outside of the win32 specific bits in pypy.rpython.lltypesystem.module.ll_os. For the most part they seem to be defined in the modules where they will be used, so it is difficult for me to understand how it all fits together. In general, a higher level description of how to handle data types which you cannot openly declare would be very helpful. Thanks in advance. -- Dan From benjamin at python.org Wed Feb 23 18:49:06 2011 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Feb 2011 11:49:06 -0600 Subject: [pypy-dev] Writing Mixed Modules for MacOS In-Reply-To: References: Message-ID: 2011/2/23 Dan Colish : > I've been working on porting the OSX specific libraries in CPython to PyPy and I'm running into issues. The shape of many fundamental data structures used in the CoreFoundation libraries are not available to me. I was hoping to just fudge it by using a ptr to that structure, but it is not working out as I had hoped. Here is a terribly incorrect sample of what I've been hacking on: http://paste.pocoo.org/show/yENK4gE11yRM9i6p46Ra and here is the corresponding osx documentation: http://goo.gl/i6mYD. To get an INT by reference you allocate an int by lltype.malloc. See some of the functions in ll_os for examples. > > I was looking at the approach taken for windows, but it is not clear to me where the fundamental structures for windows lie outside of the win32 specific bits in pypy.rpython.lltypesystem.module.ll_os. For the most part they seem to be defined in the modules where they will be used, so it is difficult for me to understand how it all fits together. In general, a higher level description of how to handle data types which you cannot openly declare would be very helpful. ://codespeak.net/mailman/listinfo/pypy-dev > -- Regards, Benjamin From thefridgeowl at gmail.com Wed Feb 23 19:04:22 2011 From: thefridgeowl at gmail.com (Henry Mason) Date: Wed, 23 Feb 2011 10:04:22 -0800 Subject: [pypy-dev] Writing Mixed Modules for MacOS In-Reply-To: References: Message-ID: <2BCB5410-64DA-437A-9146-9E3F49DB9BF9@gmail.com> On Feb 23, 2011, at 8:29 AM, Dan Colish wrote: > I've been working on porting the OSX specific libraries in CPython to PyPy and I'm running into issues. The shape of many fundamental data structures used in the CoreFoundation libraries are not available to me. I was hoping to just fudge it by using a ptr to that structure, but it is not working out as I had hoped. Here is a terribly incorrect sample of what I've been hacking on: http://paste.pocoo.org/show/yENK4gE11yRM9i6p46Ra and here is the corresponding osx documentation: http://goo.gl/i6mYD. Are you saying you don't know the shape of the opaque CFTypes, like CFNumber? Pointers to forward-declared structures is a pretty pervasive pattern in C API. For example, I see the FILE* returned by fopen() declared like this: FILEP = rffi.COpaquePtr('FILE') fopen = rffi.llexternal('fopen', [rffi.CCHARP, rffi.CCHARP], FILEP) fclose = rffi.llexternal('fclose', [FILEP], rffi.INT) in pypy/rpython/lltypesystem/test/test_ll2ctypes.py, so I imagine you'd be able to do the same for CFTypes: CFAllocatorRef = rffi.COpaquePtr('__ CFAllocator') CFNumberRef = rffi.COpaquePtr('__CFNumber') CFNumberCreateInt = rffi.llexternal('CFNumberCreate', [CFAllocatorRef, rffi.INT, rffi.VOIDP], CFNumberRef) > > I was looking at the approach taken for windows, but it is not clear to me where the fundamental structures for windows lie outside of the win32 specific bits in pypy.rpython.lltypesystem.module.ll_os. For the most part they seem to be defined in the modules where they will be used, so it is difficult for me to understand how it all fits together. In general, a higher level description of how to handle data types which you cannot openly declare would be very helpful. > > Thanks in advance. > > -- Dan > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From dcolish at gmail.com Wed Feb 23 19:48:46 2011 From: dcolish at gmail.com (Dan Colish) Date: Wed, 23 Feb 2011 10:48:46 -0800 Subject: [pypy-dev] Writing Mixed Modules for MacOS In-Reply-To: <2BCB5410-64DA-437A-9146-9E3F49DB9BF9@gmail.com> References: <2BCB5410-64DA-437A-9146-9E3F49DB9BF9@gmail.com> Message-ID: On Wednesday, February 23, 2011 at 10:04 AM, Henry Mason wrote: > On Feb 23, 2011, at 8:29 AM, Dan Colish wrote: > > > I've been working on porting the OSX specific libraries in CPython to PyPy and I'm running into issues. The shape of many fundamental data structures used in the CoreFoundation libraries are not available to me. I was hoping to just fudge it by using a ptr to that structure, but it is not working out as I had hoped. Here is a terribly incorrect sample of what I've been hacking on: http://paste.pocoo.org/show/yENK4gE11yRM9i6p46Ra and here is the corresponding osx documentation: http://goo.gl/i6mYD. > > Are you saying you don't know the shape of the opaque CFTypes, like CFNumber? Pointers to forward-declared structures is a pretty pervasive pattern in C API. For example, I see the FILE* returned by fopen() declared like this: > > FILEP = rffi.COpaquePtr('FILE') > fopen = rffi.llexternal('fopen', [rffi.CCHARP, rffi.CCHARP], FILEP) > fclose = rffi.llexternal('fclose', [FILEP], rffi.INT) > > in pypy/rpython/lltypesystem/test/test_ll2ctypes.py, so I imagine you'd be able to do the same for CFTypes: > > CFAllocatorRef = rffi.COpaquePtr('__ CFAllocator') > CFNumberRef = rffi.COpaquePtr('__CFNumber') > CFNumberCreateInt = rffi.llexternal('CFNumberCreate', [CFAllocatorRef, rffi.INT, rffi.VOIDP], CFNumberRef) > > > This is exactly what I was looking for, thank you! --Dan From holger at merlinux.eu Thu Feb 24 21:31:15 2011 From: holger at merlinux.eu (holger krekel) Date: Thu, 24 Feb 2011 21:31:15 +0100 Subject: [pypy-dev] further pypy repo migrations In-Reply-To: <4D629CF6.4010007@gmx.de> References: <20110217142305.GC30557@trillke.net> <4D629CF6.4010007@gmx.de> Message-ID: <20110224203115.GA14130@trillke.net> On Mon, Feb 21, 2011 at 18:12 +0100, Carl Friedrich Bolz wrote: > On 02/17/2011 03:23 PM, holger krekel wrote: > > There are also the repos in svn/pypy/lang that would need conversion. At > least the Smalltalk one has branches in its history, maybe it's not > important to keep the precise branch history though. OK, thanks for pointing out. > Another consideration I have is that we have been handing out external > pointers into our repositories from various places, e.g. from our blog > posts and also in some papers. I don't see an easy solution to keep them > valid, which is a bit of a pity. As posted earlier i plan to keep a read-only version of the svn repo around. We could degrade this to statically serving files or maybe use Dan's forwarder. best, holger From Ronny.Pfannschmidt at gmx.de Fri Feb 25 22:36:16 2011 From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt) Date: Fri, 25 Feb 2011 22:36:16 +0100 Subject: [pypy-dev] implementing the additional repo migrations Message-ID: <1298669776.21626.20.camel@Klappe2> hi, this night i started taking a look at the extra repos that need to be migrated. many of them contain reconstructible, but large pdf files, that i'd like to kill off for saving space. this shouldn't be a problem as long as the svn that many people link to stays available read-only, but it would need a bit of a plan if that goes as well. im currently investigating what parts of extradoc are reconstructible it seems the most costly things are * the split + full variants of the eu report (no tex aviliable) * papers (also no tex available) * api_html.tar.gz * the talks directory the talks directory is the largest size offender as well as the tree that?s the hardest to figure out (in terms of what?s reconstructible and what not). (more details later) The language repo's seem to be mostly fine, except for the prolog one, which contains a few reconstructible pdf's regards Ronny -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part URL: From lac at openend.se Sat Feb 26 03:31:31 2011 From: lac at openend.se (Laura Creighton) Date: Sat, 26 Feb 2011 03:31:31 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: Message from Ronny Pfannschmidt of "Fri, 25 Feb 2011 22:36:16 +0100." <1298669776.21626.20.camel@Klappe2> References: <1298669776.21626.20.camel@Klappe2> Message-ID: <201102260231.p1Q2VVOU018324@theraft.openend.se> In a message of Fri, 25 Feb 2011 22:36:16 +0100, Ronny Pfannschmidt writes: >hi, > >this night i started taking a look at the extra repos that need to be >migrated. > >many of them contain reconstructible, but large pdf files, that i'd like >to kill off for saving space. Why should we ever care about space? >this shouldn't be a problem as long as the svn that many people link to >stays available read-only, but it would need a bit of a plan if that >goes as well. > >im currently investigating what parts of extradoc are reconstructible I don't think that much (or maybe any) reconstruction is necessary. The things in extradoc really are the tex files, pdfs and what-have-you that you would expect from their filename extensions. The problem is that codespeak improperly serves them up as binary files. So no reconstruction needed. Just serve them properly. Laura >regards Ronny From holger at merlinux.eu Sat Feb 26 09:09:50 2011 From: holger at merlinux.eu (holger krekel) Date: Sat, 26 Feb 2011 09:09:50 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102260231.p1Q2VVOU018324@theraft.openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> Message-ID: <20110226080950.GN12505@trillke.net> Hi Laura, On Sat, Feb 26, 2011 at 03:31 +0100, Laura Creighton wrote: > In a message of Fri, 25 Feb 2011 22:36:16 +0100, Ronny Pfannschmidt writes: > >hi, > > > >this night i started taking a look at the extra repos that need to be > >migrated. > > > >many of them contain reconstructible, but large pdf files, that i'd like > >to kill off for saving space. > > Why should we ever care about space? Small repositories are faster to clone and work with. I am fine to copy everything related to extradoc, though. > >this shouldn't be a problem as long as the svn that many people link to > >stays available read-only, but it would need a bit of a plan if that > >goes as well. > > > >im currently investigating what parts of extradoc are reconstructible > > I don't think that much (or maybe any) reconstruction is necessary. > The things in extradoc really are the tex files, pdfs and what-have-you > that you would expect from their filename extensions. The problem > is that codespeak improperly serves them up as binary files. So no > reconstruction needed. Just serve them properly. this has nothing to do with codespeak but which svn:mime-type files have in the svn repository. If you find a file in the repo that should have a certain mime-type you can e. g. svn ps svn:mime-type application/pdf path/to/file.pdf it. I am actually not sure how mercurial or bitbucket handles serving such files. But I think we should anyway go for serving talks via http://pypy.org/talks/ just through apache and avoid giving out links to version control repositories. We might eventually migrate away from bitbucket and we would then again have the problem of stale links. cheers, holger From jacob at openend.se Sat Feb 26 10:25:45 2011 From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Sat, 26 Feb 2011 10:25:45 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102260231.p1Q2VVOU018324@theraft.openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> Message-ID: <201102261025.46282.jacob@openend.se> Saturday 26 February 2011 you wrote: > In a message of Fri, 25 Feb 2011 22:36:16 +0100, Ronny Pfannschmidt writes: > >hi, > > > >this night i started taking a look at the extra repos that need to be > >migrated. > > > >many of them contain reconstructible, but large pdf files, that i'd like > >to kill off for saving space. > > Why should we ever care about space? A mercurial repository is cloned as a whole, including all revision history. This does take up a lot of space with text files, but produces very large repositories with binary formats, resulting in bad behaviour on slow links and problems on devices with limited space. While I am fine with dropping older revisions of just about everything in extradoc, I wonder if it wouldn't be better better for the future to keep this repository in svn format. That way you will only get one copy of everything when cloning the repository. Jacob -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: From lac at openend.se Sat Feb 26 10:29:47 2011 From: lac at openend.se (Laura Creighton) Date: Sat, 26 Feb 2011 10:29:47 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: Message from holger krekel of "Sat, 26 Feb 2011 09:09:50 +0100." <20110226080950.GN12505@trillke.net> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <20110226080950.GN12505@trillke.net> Message-ID: <201102260929.p1Q9TlRf022670@theraft.openend.se> In a message of Sat, 26 Feb 2011 09:09:50 +0100, holger krekel writes: >Hi Laura, > >> Why should we ever care about space? > >Small repositories are faster to clone and work with. >I am fine to copy everything related to extradoc, though. I thought we had unlimited repos in bitbucket. If so, then this is an argument for putting the extra documentation in a different repo than the pypy one, not for getting rid of things in extradoc. >> I don't think that much (or maybe any) reconstruction is necessary. >> The things in extradoc really are the tex files, pdfs and what-have-you >> that you would expect from their filename extensions. The problem >> is that codespeak improperly serves them up as binary files. So no >> reconstruction needed. Just serve them properly. > >this has nothing to do with codespeak but which svn:mime-type >files have in the svn repository. If you find a file in the repo >that should have a certain mime-type you can e. g. > > svn ps svn:mime-type application/pdf path/to/file.pdf > >it. > >I am actually not sure how mercurial or bitbucket handles serving such fi >les. Sorry, I knew that, I didn't mean to imply that it was some flaw in codespeak. Though it is very far from clear to me how svn got in the business of giving our files mimetypes, which I think happened secretly, under-the-hood. I don't think any of us explicitly went around typing svn commands to set a mime-type of binary to those files, it just happened silently. And, for me, at any rate, svn was an unexpected culprit -- by this I mean that when I went investigating the problem of 'why is this talk being served up as a binary file' svn was not one of the places I thought to look. The mercurial authors (like me) don't like the concept of binrary files, see http://mercurial.selenic.com/wiki/BinaryFiles , so I guess all we need to find out is does bitbucket itself make assumptions. I wouldn't think so, but then I thought that about svn as well. >But I think we should anyway go for serving talks via http://pypy.org/tal >ks/ >just through apache and avoid giving out links to version control >repositories. We might eventually migrate away from bitbucket >and we would then again have the problem of stale links. I'm not fond of a solution which leaves the extradoc files in two versions, the one in the repo and the one on http://pypy.org/talks , and where you have to keep remembering to change the pypy.org one because you have made a change to the one in the repo. >cheers, >holger I think we may have a more general problem, in that the flexibility of pypy should lead to massive experimentation, not only by ourselves but by third parties. What then should we do with the experiments? I'd like to find a way to keep the interesting ones around without making branching horribly slow. I thought that having many separate repositories would go a long way to getting this, was this also a mistaken assumption? Laura From alex.gaynor at gmail.com Sat Feb 26 10:29:20 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 26 Feb 2011 04:29:20 -0500 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102261025.46282.jacob@openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> Message-ID: On Sat, Feb 26, 2011 at 4:25 AM, Jacob Hall?n wrote: > Saturday 26 February 2011 you wrote: > > In a message of Fri, 25 Feb 2011 22:36:16 +0100, Ronny Pfannschmidt > writes: > > >hi, > > > > > >this night i started taking a look at the extra repos that need to be > > >migrated. > > > > > >many of them contain reconstructible, but large pdf files, that i'd like > > >to kill off for saving space. > > > > Why should we ever care about space? > > A mercurial repository is cloned as a whole, including all revision > history. > This does take up a lot of space with text files, but produces very large > repositories with binary formats, resulting in bad behaviour on slow links > and > problems on devices with limited space. > > While I am fine with dropping older revisions of just about everything in > extradoc, I wonder if it wouldn't be better better for the future to keep > this > repository in svn format. That way you will only get one copy of everything > when cloning the repository. > > Jacob > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > There's also http://mercurial.selenic.com/wiki/BigfilesExtension, which I know nothing about. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From lac at openend.se Sat Feb 26 10:51:34 2011 From: lac at openend.se (Laura Creighton) Date: Sat, 26 Feb 2011 10:51:34 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: Message from Jacob =?iso-8859-1?q?Hall=E9n?= of "Sat, 26 Feb 2011 10:25:45 +0100." <201102261025.46282.jacob@openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> Message-ID: <201102260951.p1Q9pYQ9024873@theraft.openend.se> In a message of Sat, 26 Feb 2011 10:25:45 +0100, Jacob Hall?n writes: >While I am fine with dropping older revisions of just about everything in >extradoc, I wonder if it wouldn't be better better for the future to keep >this repository in svn format. That way you will only get one copy of >everything when cloning the repository. > >Jacob I'm not fine with the dropping of older revisions. One of the chief benefits for me of moving to a version control system was that I could feel comfortable ruthlessly condensing my writing, knowing that if I ever wanted this stuff later -- say to use in a different document, I could always go back and get the old revision that contained the wonderful words or diagrams I now propose to cut. And this has happened in the past, where early versions of things I wrote ended up raised out of the grave of the repository to live on as part of completely different documents. I'm not going to be comfortable deleting stuff this if I think that the grim reaper is out there, just waiting to purge all my earlier attempts once some document is deemed to be 'final'. So I either won't delete stuff, or I will go back to my old practice of having dozens of versions around 'just in case'. I'm fine with continuing to have the extradoc managed by svn, though I really want a script that runs nightly looking for things in extradoc that have a mimetype of binary and complains about this. Laura From fijall at gmail.com Sat Feb 26 10:54:39 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 26 Feb 2011 11:54:39 +0200 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102260951.p1Q9pYQ9024873@theraft.openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> <201102260951.p1Q9pYQ9024873@theraft.openend.se> Message-ID: On Sat, Feb 26, 2011 at 11:51 AM, Laura Creighton wrote: > In a message of Sat, 26 Feb 2011 10:25:45 +0100, Jacob Hall?n writes: > > > >>While I am fine with dropping older revisions of just about everything in >>extradoc, I wonder if it wouldn't be better better for the future to keep >>this repository in svn format. That way you will only get one copy of >>everything when cloning the repository. >> >>Jacob > > I'm not fine with the dropping of older revisions. ?One of the chief > benefits for me of moving to a version control system was that I could > feel comfortable ruthlessly condensing my writing, knowing that if I > ever wanted this stuff later -- say to use in a different document, > I could always go back and get the old revision that contained the > wonderful words or diagrams I now propose to cut. ?And this has > happened in the past, where early versions of things I wrote ended > up raised out of the grave of the repository to live on as part of > completely different documents. > > I'm not going to be comfortable deleting stuff this if I think that > the grim reaper is out there, just waiting to purge all my earlier > attempts once some document is deemed to be 'final'. ?So I either > won't delete stuff, or I will go back to my old practice of having > dozens of versions around 'just in case'. > > I'm fine with continuing to have the extradoc managed by svn, though > I really want a script that runs nightly looking for things in > extradoc that have a mimetype of binary and complains about this. > > Laura > Isn't the debate mostly about older revisions of binary files, since the rest is fine? Or you want to keep those as well? (say .doc) From lac at openend.se Sat Feb 26 10:59:09 2011 From: lac at openend.se (Laura Creighton) Date: Sat, 26 Feb 2011 10:59:09 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: Message from Maciej Fijalkowski of "Sat, 26 Feb 2011 11:54:39 +0200." References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> <201102260951.p1Q9pYQ9024873@theraft.openend.se> Message-ID: <201102260959.p1Q9x9jT025385@theraft.openend.se> In a message of Sat, 26 Feb 2011 11:54:39 +0200, Maciej Fijalkowski writes: >Isn't the debate mostly about older revisions of binary files, since >the rest is fine? Or you want to keep those as well? (say .doc) I don't care about the old versions of binary files. Laura From pgiarrusso at mathematik.uni-marburg.de Sat Feb 26 13:01:41 2011 From: pgiarrusso at mathematik.uni-marburg.de (Paolo Giarrusso) Date: Sat, 26 Feb 2011 13:01:41 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102260929.p1Q9TlRf022670@theraft.openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <20110226080950.GN12505@trillke.net> <201102260929.p1Q9TlRf022670@theraft.openend.se> Message-ID: On Sat, Feb 26, 2011 at 10:29, Laura Creighton wrote: > I think we may have a more general problem, in that the flexibility of > pypy should lead to massive experimentation, not only by ourselves but > by third parties. ?What then should we do with the experiments? ?I'd like > to find a way to keep the interesting ones around without making > branching horribly slow. ?I thought that having many separate repositories > would go a long way to getting this, was this also a mistaken assumption? DVCS allow you to give a different meaning to "many separate repositories", which was not possible with SVN and which answers your question. Namely, there is little reason why experiment branches should be part of the same repo - you can just fork the project through bitbucket tools and make changes on the fork. If and when the fork is completed and useful, it can be pulled back, otherwise it just stays separate. With Hg the only problem is in storing N repositories on a computer without storing N copies of the common history, which is a different but related problem. To make branching fast, hardlinks allow to share the history between related repos, especially in Git but it seems also in Hg - search "hardlink" in the hg(1) man page, and this extension: http://mercurial.selenic.com/wiki/RelinkExtension While I'm a fan of Hg, the Git repository layout seems intrinsically better from this point of view: with Hg, as soon as you commit a change to a file in a branch, the existing history of that file can't be shared any more between the two repos (at least, it can't possibly be shared through a hardlink; it doesn't seem they implement other solutions, and it would be hard to do that in a way coherent with their design). I've got no experience with Hg on huge projects, while I have 8 branches of the Linux kernel laying around: the history of each additional branches takes ~5M (compared to 450M for the history of the main repo). If the branch stays close enough, though, this might not be a problem, but YMMV. Cheers, -- Paolo Giarrusso - Ph.D. Student http://www.informatik.uni-marburg.de/~pgiarrusso/ From arigo at tunes.org Sat Feb 26 13:03:15 2011 From: arigo at tunes.org (Armin Rigo) Date: Sat, 26 Feb 2011 13:03:15 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <201102260959.p1Q9x9jT025385@theraft.openend.se> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> <201102260951.p1Q9pYQ9024873@theraft.openend.se> <201102260959.p1Q9x9jT025385@theraft.openend.se> Message-ID: Hi Laura, On Sat, Feb 26, 2011 at 10:59 AM, Laura Creighton wrote: > I don't care about the old versions of binary files. That was the only thing we talked about -- as far as I understood, it was never suggested that we should stop tracking revisions of .txt or .tex files. I don't know the BigfilesExtension either, but it looks to me like we can achieve some more precise result manually. Something along the lines of: the .pdf's built from .tex's are not checked in, but they are in some standardized place on http://pypy.org, where we can fetch them, update them (via ssh), or point people to (via their url). This can be easily done with a script independent from Mercurial. (The point is of course that tracking revisions is a bit useless, because we can always go back in time and re-run latex2pdf.) Well, this was my 2 cents to this discussion, but maybe Ronny is just worrying too much. I don't think we care that much about saving space or transfer time. It's anyway not like everybody on the planet should download our extradoc repository. A bient?t, Armin. From cfbolz at gmx.de Sat Feb 26 22:17:21 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 26 Feb 2011 22:17:21 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <1298669776.21626.20.camel@Klappe2> References: <1298669776.21626.20.camel@Klappe2> Message-ID: <4D696DE1.7020906@gmx.de> On 02/25/2011 10:36 PM, Ronny Pfannschmidt wrote: > this night i started taking a look at the extra repos that need to be > migrated. > > many of them contain reconstructible, but large pdf files, that i'd like > to kill off for saving space. > > this shouldn't be a problem as long as the svn that many people link to > stays available read-only, but it would need a bit of a plan if that > goes as well. > > im currently investigating what parts of extradoc are reconstructible > > > it seems the most costly things are > > * the split + full variants of the eu report (no tex aviliable) > * papers (also no tex available) > * api_html.tar.gz > * the talks directory > > the talks directory is the largest size offender as well as the tree > that?s the hardest to figure out (in terms of what?s reconstructible and > what not). > > (more details later) > > > The language repo's seem to be mostly fine, > except for the prolog one, which contains a few reconstructible pdf's Don't worry about Prolog, that has been manually migrated already by me a while ago. Carl Friedrich From cfbolz at gmx.de Sat Feb 26 22:25:05 2011 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 26 Feb 2011 22:25:05 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> <201102260951.p1Q9pYQ9024873@theraft.openend.se> <201102260959.p1Q9x9jT025385@theraft.openend.se> Message-ID: <4D696FB1.5020100@gmx.de> On 02/26/2011 01:03 PM, Armin Rigo wrote: > Hi Laura, > > On Sat, Feb 26, 2011 at 10:59 AM, Laura Creighton wrote: >> I don't care about the old versions of binary files. > > That was the only thing we talked about -- as far as I understood, it > was never suggested that we should stop tracking revisions of .txt or > .tex files. I don't know the BigfilesExtension either, but it looks > to me like we can achieve some more precise result manually. > Something along the lines of: the .pdf's built from .tex's are not > checked in, but they are in some standardized place on > http://pypy.org, where we can fetch them, update them (via ssh), or > point people to (via their url). This can be easily done with a > script independent from Mercurial. (The point is of course that > tracking revisions is a bit useless, because we can always go back in > time and re-run latex2pdf.) Not necessarily, it's always possible that whatever latex packages were needed to compile the pdf are no longer around or a big hassle to install. This can make regeneration impractical. So I am in favor of keeping the PDFs in the repo. Carl Friedrich From lac at openend.se Sun Feb 27 16:38:03 2011 From: lac at openend.se (Laura Creighton) Date: Sun, 27 Feb 2011 16:38:03 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: Message from Laura Creighton of "Sun, 27 Feb 2011 14:59:46 +0100." <201102271359.p1RDxkpX012065@theraft.openend.se> References: <201102271359.p1RDxkpX012065@theraft.openend.se> Message-ID: <201102271538.p1RFc3rG020721@theraft.openend.se> I'd like to change what is displayed on the speed.python.org frontpage. Right now, as I look at http://speed.pypy.org/ I see, under a section called 'latest results' a list of all the recent times when we became slower. It's thus a 'recent problems' page -- we have actually improved in recent times in many areas, and nowhere is that shown. As we go off to PyCON, which is March 9-17, I intend to mention how great PyPy is, and that you can see it for yourself at speed.pypy.org. Thus, without lying, I would like it if the first impression of PyPy's speed that people got when looking at the site was 'we're getting faster'. Do you think you could change the front page so that what was displayed was more balanced with respect to good news and bad news? I realise that there is nothing you can do if we make a recent build that slows everything down, but for instance in build 42312:392b (Feb 26) we have improvments which are not shown on the main page. I actually think that the _trend_ is a more useful thing to display on the front page, though that might be because it is so green right now. :-) The other thing I want is for the graphs you get, for instance with http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit to have, in addition to the selection button beside: 'result for revision' an actual label that says 'build 42312:392b' or something that you can select with your mouse and use to paste into things like this mail article. It would also be useful to label the run with something more meaningful than 'tannit' for outsiders -- 64 bit ubuntu linux for instance. Thanks very much, Laura Creighton From anto.cuni at gmail.com Sun Feb 27 18:52:44 2011 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sun, 27 Feb 2011 18:52:44 +0100 Subject: [pypy-dev] implementing the additional repo migrations In-Reply-To: <4D696FB1.5020100@gmx.de> References: <1298669776.21626.20.camel@Klappe2> <201102260231.p1Q2VVOU018324@theraft.openend.se> <201102261025.46282.jacob@openend.se> <201102260951.p1Q9pYQ9024873@theraft.openend.se> <201102260959.p1Q9x9jT025385@theraft.openend.se> <4D696FB1.5020100@gmx.de> Message-ID: +1 on keeping the pdf files. It happens quite often not to have all the needed latex packages On 2/26/11, Carl Friedrich Bolz wrote: > On 02/26/2011 01:03 PM, Armin Rigo wrote: >> Hi Laura, >> >> On Sat, Feb 26, 2011 at 10:59 AM, Laura Creighton wrote: >>> I don't care about the old versions of binary files. >> >> That was the only thing we talked about -- as far as I understood, it >> was never suggested that we should stop tracking revisions of .txt or >> .tex files. I don't know the BigfilesExtension either, but it looks >> to me like we can achieve some more precise result manually. >> Something along the lines of: the .pdf's built from .tex's are not >> checked in, but they are in some standardized place on >> http://pypy.org, where we can fetch them, update them (via ssh), or >> point people to (via their url). This can be easily done with a >> script independent from Mercurial. (The point is of course that >> tracking revisions is a bit useless, because we can always go back in >> time and re-run latex2pdf.) > > Not necessarily, it's always possible that whatever latex packages were > needed to compile the pdf are no longer around or a big hassle to > install. This can make regeneration impractical. So I am in favor of > keeping the PDFs in the repo. > > Carl Friedrich > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From tobami at googlemail.com Sun Feb 27 19:36:52 2011 From: tobami at googlemail.com (Miquel Torres) Date: Sun, 27 Feb 2011 19:36:52 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: <201102271538.p1RFc3rG020721@theraft.openend.se> References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> Message-ID: Hi Laura, you bring up good points, however, it is not as straight forward as it seems. >I see, under a section > called 'latest results' a list of all the recent times when we became > slower. ?It's thus a 'recent problems' page -- we have actually improved > in recent times in many areas, and nowhere is that shown. Well, it really is a list of the latest results. The problem is that speed.pypy.org is foremost a tool to help in development. As such, the logic behind the "latest results" list is regression oriented, or let us say pessimistic. For example, this revision: http://speed.pypy.org/changes/?rev=42312:392bbf936179 The average change is actually -0.91%, which is actually an improvement, though not an statistical significant one. However, There was a sizeable regression in spitfire_cstringio, +5.21. The "summary" for that revision is then "regression for an individual benchmark". Which is actually what developers need to know: they should check whether that revision really introduced a real regression in performance. Now, it is true that since that new main page was introduced the impression it has given is one of regressions, mostly. But if you look at all the graphs together 8http://speed.pypy.org/timeline), you see that in the last weeks there has been a slight upwards (worse performance) trend in many benchmarks. So I think Codespeed has done the right thing! Note that a week ago I did up the threshold from 3% to 4% changes That said, I do understand where you are coming from. I would point outsiders though directly to http://speed.pypy.org/comparison/ So what can some body think about what could be changed or added so that the main page doesn't give a negative impression to the non-developer? Something I could think of is to add, above the results list, a plot showing the overall trend over the last 2 or 3 months. What do you think? > The other thing I want is for the graphs you get, for instance with > http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit > to have, in addition to the selection button beside: 'result for revision' > an actual label that says 'build 42312:392b' or something that you > can select with your mouse and use to paste into things like this mail > article. I think to the right of the changes table there is a box with info for the revision, with a text field you can select and copy. Isn't that what you want? > article. ?It would also be useful to label the run with something more > meaningful than 'tannit' for outsiders -- 64 bit ubuntu linux ?for instance. Agreed. I would also rather call the machines/environments like that. 2011/2/27 Laura Creighton : > > ?sorry Miguel.> > > I'd like to change what is displayed on the speed.python.org frontpage. > Right now, as I look at http://speed.pypy.org/ ?I see, under a section > called 'latest results' a list of all the recent times when we became > slower. ?It's thus a 'recent problems' page -- we have actually improved > in recent times in many areas, and nowhere is that shown. ?As we go off > to PyCON, which is March 9-17, I intend to mention how great PyPy is, > and that you can see it for yourself at speed.pypy.org. ?Thus, without > lying, I would like it if the first impression of PyPy's speed that > people got when looking at the site was 'we're getting faster'. ?Do you > think you could change the front page so that what was displayed was > more balanced with respect to good news and bad news? > > I realise that there is nothing you can do if we make a recent build > that slows everything down, but for instance in build 42312:392b (Feb 26) > we have improvments which are not shown on the main page. ?I actually > think that the _trend_ ?is a more useful thing to display on the front page, > though that might be because it is so green right now. :-) > > The other thing I want is for the graphs you get, for instance with > http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit > to have, in addition to the selection button beside: 'result for revision' > an actual label that says 'build 42312:392b' or something that you > can select with your mouse and use to paste into things like this mail > article. ?It would also be useful to label the run with something more > meaningful than 'tannit' for outsiders -- 64 bit ubuntu linux ?for instance. > > Thanks very much, > Laura Creighton > From lac at openend.se Sun Feb 27 20:11:39 2011 From: lac at openend.se (Laura Creighton) Date: Sun, 27 Feb 2011 20:11:39 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: Message from Miquel Torres of "Sun, 27 Feb 2011 19:36:52 +0100." References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> Message-ID: <201102271911.p1RJBdOg007318@theraft.openend.se> In a message of Sun, 27 Feb 2011 19:36:52 +0100, Miquel Torres writes: >Hi Laura, > >you bring up good points, however, it is not as straight forward as it se >em= >Well, it really is a list of the latest results. The problem is that >speed.pypy.org is foremost a tool to help in development. As such, the >logic behind the "latest results" list is regression oriented, or let >us say pessimistic. > >For example, this revision: >http://speed.pypy.org/changes/?rev=3D42312:392bbf936179 > >The average change is actually -0.91%, which is actually an >improvement, though not an statistical significant one. However, There >was a sizeable regression in spitfire_cstringio, +5.21. The "summary" >for that revision is then "regression for an individual benchmark". >Which is actually what developers need to know: they should check >whether that revision really introduced a real regression in >performance. I understand this. I just don't think that this should be on our front page. Developers can get used to looking anywhere, and indeed I never use the front page for looking at anything important -- I am always looking at the complete stats for any runs. I think that speed.pypy.org -- the front page -- should primarily be of use to people who want to find out if pypy is good for them, people who want to convince their bosses that they should switch, and people who just want to cheer us on and add to the general warm feeling about pypy. I see it as the prime tool in the world domination project. >That said, I do understand where you are coming from. I would point >outsiders though directly to http://speed.pypy.org/comparison/ This is the wrong way to do things from a usability point of view. What the casual person wants is not a way to dig down and get information, but something already packaged for them which already tells them the main story. Then they can dig down if they actually care. This is what Steve Krug calls the 'Don't Make Me Think' principle. >So what can some body think about what could be changed or added so >that the main page doesn't give a negative impression to the >non-developer? > >Something I could think of is to add, above the results list, a plot >showing the overall trend over the last 2 or 3 months. What do you >think? I think this would be a very nice thing to have as the home page for speed.pypy.org Then the page that we have now could be called something like http://speed.pypy.org/regressions or something. >> The other thing I want is for the graphs you get, for instance with >> http://speed.pypy.org/changes/?rev=3D42312:392bbf936179&exe=3D%203&env= >3D= >tannit >> to have, in addition to the selection button beside: 'result for revisi >on= >' >> an actual label that says 'build 42312:392b' or something that you >> can select with your mouse and use to paste into things like this mail >> article. > >I think to the right of the changes table there is a box with info for >the revision, with a text field you can select and copy. Isn't that >what you want? Looking at http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit There is nothing to the right of the table. To the right of the label that says: Results for revision is a box that you can use to select different builds to look at. The text in this box is not selectable; you cannot paste it anywhere. To the right of that is a link that says 'Permalink'. As far as I can tell clicking it causes the page to refresh and nothing more. This is with iceweasel 3.5.16 (which is debian's repackage of firefox 3.5.16) Laura From tobami at googlemail.com Sun Feb 27 20:22:57 2011 From: tobami at googlemail.com (Miquel Torres) Date: Sun, 27 Feb 2011 20:22:57 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: <201102271911.p1RJBdOg007318@theraft.openend.se> References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> Message-ID: Ok, I'll try that then. > There is nothing to the right of the table. What is your screen width? The Revision info box should be below the table if your width is small. 2011/2/27 Laura Creighton : > In a message of Sun, 27 Feb 2011 19:36:52 +0100, Miquel Torres writes: >>Hi Laura, >> >>you bring up good points, however, it is not as straight forward as it se >>em= > >>Well, it really is a list of the latest results. The problem is that >>speed.pypy.org is foremost a tool to help in development. As such, the >>logic behind the "latest results" list is regression oriented, or let >>us say pessimistic. >> >>For example, this revision: >>http://speed.pypy.org/changes/?rev=3D42312:392bbf936179 >> >>The average change is actually -0.91%, which is actually an >>improvement, though not an statistical significant one. However, There >>was a sizeable regression in spitfire_cstringio, +5.21. The "summary" >>for that revision is then "regression for an individual benchmark". >>Which is actually what developers need to know: they should check >>whether that revision really introduced a real regression in >>performance. > > I understand this. ?I just don't think that this should be on our > front page. ? Developers can get used to looking anywhere, and indeed > I never use the front page for looking at anything important -- I am > always looking at the complete stats for any runs. > > I think that speed.pypy.org -- the front page -- should primarily be > of use to people who want to find out if pypy is good for them, people > who want to convince their bosses that they should switch, and people > who just want to cheer us on and add to the general warm feeling > about pypy. ?I see it as the prime tool in the world domination > project. > >>That said, I do understand where you are coming from. I would point >>outsiders though directly to http://speed.pypy.org/comparison/ > > This is the wrong way to do things from a usability point of view. > What the casual person wants is not a way to dig down and get information, > but something already packaged for them which already tells them > the main story. ?Then they can dig down if they actually care. ?This is > what Steve Krug calls the 'Don't Make Me Think' principle. > >>So what can some body think about what could be changed or added so >>that the main page doesn't give a negative impression to the >>non-developer? >> >>Something I could think of is to add, above the results list, a plot >>showing the overall trend over the last 2 or 3 months. What do you >>think? > > I think this would be a very nice thing to have as the home page for > speed.pypy.org ?Then the page that we have now could be called something > like http://speed.pypy.org/regressions or something. > >>> The other thing I want is for the graphs you get, for instance with >>> http://speed.pypy.org/changes/?rev=3D42312:392bbf936179&exe=3D%203&env= >>3D= >>tannit >>> to have, in addition to the selection button beside: 'result for revisi >>on= >>' >>> an actual label that says 'build 42312:392b' or something that you >>> can select with your mouse and use to paste into things like this mail >>> article. >> >>I think to the right of the changes table there is a box with info for >>the revision, with a text field you can select and copy. Isn't that >>what you want? > > Looking at http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit > > There is nothing to the right of the table. ?To the right of the label > that says: Results for revision ?is a box that you can use to select different > builds to look at. ?The text in this box is not selectable; you cannot > paste it anywhere. ?To the right of that is a link that says 'Permalink'. > As far as I can tell clicking it causes the page to refresh and nothing > more. > > This is with iceweasel 3.5.16 (which is debian's repackage of firefox > 3.5.16) > > Laura > > From fijall at gmail.com Sun Feb 27 20:57:33 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sun, 27 Feb 2011 21:57:33 +0200 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> Message-ID: Hey. Just my 5 cents. It would be cool if default view has a down-scaled version of comparison against CPython. I can look anywhere for recent changes. Also the recent changes as they're now are not very informative and I don't use them at all. They stick around, so I don't know if they're new or old. I'm also as interested in good as in bad changes. Simply this: http://speed.pypy.org/changes/ is way more informative. Can we either just remove the red recent changes for now or simply put a vs cpython, scaled down graph there? At least for pycon this seems like a better way to go. Cheers, fijal PS. Miquel, don't get me wrong, I think you're doing an awesome job, the speed website itself was a huge step forward for us. From lac at openend.se Sun Feb 27 21:04:36 2011 From: lac at openend.se (Laura Creighton) Date: Sun, 27 Feb 2011 21:04:36 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: Message from Miquel Torres of "Sun, 27 Feb 2011 20:22:57 +0100." References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> Message-ID: <201102272004.p1RK4aeB011714@theraft.openend.se> In a message of Sun, 27 Feb 2011 20:22:57 +0100, Miquel Torres writes: >Ok, I'll try that then. > >> There is nothing to the right of the table. >What is your screen width? >The Revision info box should be below the table if your width is small. xpdyinfo says dimensions: 1024x768 pixels (270x203 millimeters) Laura From chef at ghum.de Sun Feb 27 21:06:07 2011 From: chef at ghum.de (Massa, Harald Armin) Date: Sun, 27 Feb 2011 21:06:07 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> Message-ID: Adding to Maciej's 5cents: > It would be cool if default view has a down-scaled version of > comparison against CPython. I for one second this suggestion. Comparison against cPython is the TLDR of speed.pypy.org; the abstract, the executive-level-information. Harald -- GHUM GmbH Harald Armin Massa Spielberger Stra?e 49 70435 Stuttgart 0173/9409607 Amtsgericht Stuttgart, HRB 734971 - persuadere. et programmare From lac at openend.se Sun Feb 27 21:08:37 2011 From: lac at openend.se (Laura Creighton) Date: Sun, 27 Feb 2011 21:08:37 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: Message from Maciej Fijalkowski of "Sun, 27 Feb 2011 21:57:33 +0200." References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> Message-ID: <201102272008.p1RK8bh7012202@theraft.openend.se> In a message of Sun, 27 Feb 2011 21:57:33 +0200, Maciej Fijalkowski writes: >Hey. > >Just my 5 cents. > >It would be cool if default view has a down-scaled version of >comparison against CPython. I can look anywhere for recent changes. >Also the recent changes as they're now are not very informative and I >don't use them at all. They stick around, so I don't know if they're >new or old. I'm also as interested in good as in bad changes. Simply >this: http://speed.pypy.org/changes/ is way more informative. > >Can we either just remove the red recent changes for now or simply put >a vs cpython, scaled down graph there? > >At least for pycon this seems like a better way to go. > >Cheers, >fijal > >PS. Miquel, don't get me wrong, I think you're doing an awesome job, >the speed website itself was a huge step forward for us. This sounds good to me as well, and I too don't want Miquel to think that I am ungrateful for all his hard work. The site is really good for us, and thank you. Laura From tobami at googlemail.com Sun Feb 27 23:00:38 2011 From: tobami at googlemail.com (Miquel Torres) Date: Sun, 27 Feb 2011 23:00:38 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: <201102272008.p1RK8bh7012202@theraft.openend.se> References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> <201102272008.p1RK8bh7012202@theraft.openend.se> Message-ID: Hi all, hey, it's not like I crumble at the sound of the slightest criticism ;-) Open Source is about this. Ideas, feedback, etc. Only then can we get better! I think I can draw two conclusions: - For Pycon, and maybe permanently, a general comparison against CPython or an overall timeline would be better. I see I have about 1 week to come up with something. - If Maciej, who obviously is a developer, doesn't think that the report, or "recent results" feature is useful, that means that at the very least it needs tweaking. I will think and experiment with different possibilities. For one I will increase the threshold again, to 5 o 6% so that there is a higher signal to noise ratio. I can also imagine showing two numbers: one for the biggest improvement and one for the biggest regression. And maybe the average. We'll see. If anyone has a clear idea of how that could become more useful please share. Cheers, Miquel 2011/2/27 Laura Creighton : > In a message of Sun, 27 Feb 2011 21:57:33 +0200, Maciej Fijalkowski writes: >>Hey. >> >>Just my 5 cents. >> >>It would be cool if default view has a down-scaled version of >>comparison against CPython. I can look anywhere for recent changes. >>Also the recent changes as they're now are not very informative and I >>don't use them at all. They stick around, so I don't know if they're >>new or old. I'm also as interested in good as in bad changes. Simply >>this: http://speed.pypy.org/changes/ is way more informative. >> >>Can we either just remove the red recent changes for now or simply put >>a vs cpython, scaled down graph there? >> >>At least for pycon this seems like a better way to go. >> >>Cheers, >>fijal >> >>PS. Miquel, don't get me wrong, I think you're doing an awesome job, >>the speed website itself was a huge step forward for us. > > This sounds good to me as well, and I too don't want Miquel to think > that I am ungrateful for all his hard work. ?The site is really good > for us, and thank you. > > Laura > From fijall at gmail.com Mon Feb 28 07:04:47 2011 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 28 Feb 2011 08:04:47 +0200 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> <201102272008.p1RK8bh7012202@theraft.openend.se> Message-ID: On Mon, Feb 28, 2011 at 12:00 AM, Miquel Torres wrote: > Hi all, > > hey, it's not like I crumble at the sound of the slightest criticism ;-) > Open Source is about this. Ideas, feedback, etc. Only then can we get better! > > I think I can draw two conclusions:"I > - For Pycon, and maybe permanently, a general comparison against > CPython or an overall timeline would be better. I see I have about 1 > week to come up with something. > - If Maciej, who obviously is a developer, doesn't think that the > report, or "recent results" feature is useful, that means that at the > very least it needs tweaking. I will think and experiment with > different possibilities. For one I will increase the threshold again, > to 5 o 6% so that there is a higher signal to noise ratio. I can also > imagine showing two numbers: one for the biggest improvement and one > for the biggest regression. And maybe the average. We'll see. If > anyone has a clear idea of how that could become more useful please > share. My biggest issue is with the fact that they show up and just stay. This means there is no way to distinguish between "I've seen this" or "I didn't see this" which make it useless. > > Cheers, > Miquel > > > 2011/2/27 Laura Creighton : >> In a message of Sun, 27 Feb 2011 21:57:33 +0200, Maciej Fijalkowski writes: >>>Hey. >>> >>>Just my 5 cents. >>> >>>It would be cool if default view has a down-scaled version of >>>comparison against CPython. I can look anywhere for recent changes. >>>Also the recent changes as they're now are not very informative and I >>>don't use them at all. They stick around, so I don't know if they're >>>new or old. I'm also as interested in good as in bad changes. Simply >>>this: http://speed.pypy.org/changes/ is way more informative. >>> >>>Can we either just remove the red recent changes for now or simply put >>>a vs cpython, scaled down graph there? >>> >>>At least for pycon this seems like a better way to go. >>> >>>Cheers, >>>fijal >>> >>>PS. Miquel, don't get me wrong, I think you're doing an awesome job, >>>the speed website itself was a huge step forward for us. >> >> This sounds good to me as well, and I too don't want Miquel to think >> that I am ungrateful for all his hard work. ?The site is really good >> for us, and thank you. >> >> Laura >> > From tobami at googlemail.com Mon Feb 28 08:33:16 2011 From: tobami at googlemail.com (Miquel Torres) Date: Mon, 28 Feb 2011 08:33:16 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> <201102271911.p1RJBdOg007318@theraft.openend.se> <201102272008.p1RK8bh7012202@theraft.openend.se> Message-ID: Well, that use-case is covered by the RSS feed ;-) 2011/2/28 Maciej Fijalkowski : > On Mon, Feb 28, 2011 at 12:00 AM, Miquel Torres wrote: >> Hi all, >> >> hey, it's not like I crumble at the sound of the slightest criticism ;-) >> Open Source is about this. Ideas, feedback, etc. Only then can we get better! >> >> I think I can draw two conclusions:"I >> - For Pycon, and maybe permanently, a general comparison against >> CPython or an overall timeline would be better. I see I have about 1 >> week to come up with something. >> - If Maciej, who obviously is a developer, doesn't think that the >> report, or "recent results" feature is useful, that means that at the >> very least it needs tweaking. I will think and experiment with >> different possibilities. For one I will increase the threshold again, >> to 5 o 6% so that there is a higher signal to noise ratio. I can also >> imagine showing two numbers: one for the biggest improvement and one >> for the biggest regression. And maybe the average. We'll see. If >> anyone has a clear idea of how that could become more useful please >> share. > > My biggest issue is with the fact that they show up and just stay. > This means there is no way to distinguish between "I've seen this" or > "I didn't see this" which make it useless. > > >> >> Cheers, >> Miquel >> >> >> 2011/2/27 Laura Creighton : >>> In a message of Sun, 27 Feb 2011 21:57:33 +0200, Maciej Fijalkowski writes: >>>>Hey. >>>> >>>>Just my 5 cents. >>>> >>>>It would be cool if default view has a down-scaled version of >>>>comparison against CPython. I can look anywhere for recent changes. >>>>Also the recent changes as they're now are not very informative and I >>>>don't use them at all. They stick around, so I don't know if they're >>>>new or old. I'm also as interested in good as in bad changes. Simply >>>>this: http://speed.pypy.org/changes/ is way more informative. >>>> >>>>Can we either just remove the red recent changes for now or simply put >>>>a vs cpython, scaled down graph there? >>>> >>>>At least for pycon this seems like a better way to go. >>>> >>>>Cheers, >>>>fijal >>>> >>>>PS. Miquel, don't get me wrong, I think you're doing an awesome job, >>>>the speed website itself was a huge step forward for us. >>> >>> This sounds good to me as well, and I too don't want Miquel to think >>> that I am ungrateful for all his hard work. ?The site is really good >>> for us, and thank you. >>> >>> Laura >>> >> > From lac at openend.se Mon Feb 28 08:44:15 2011 From: lac at openend.se (Laura Creighton) Date: Mon, 28 Feb 2011 08:44:15 +0100 Subject: [pypy-dev] Change to the frontpage of speed.pypy.org In-Reply-To: Message from Miquel Torres of "Sun, 27 Feb 2011 19:36:52 +0100." References: <201102271359.p1RDxkpX012065@theraft.openend.se> <201102271538.p1RFc3rG020721@theraft.openend.se> Message-ID: <201102280744.p1S7iFh7009804@theraft.openend.se> Also, can you change every usage of the word 'less' on the axis of the graphs to 'smaller'. As far as I can tell, every time it is an incorrect and ungramatically usage of English. Sometimes the legend is 'seconds - less is better'. But seconds are inherantly countable, so it is incorrect English to say 'there are less seconds' -- rather that there are *fewer* seconds. Or you could change that legend to read 'time in seconds' -- because time is considered uncountable, so you have more or less time, not more or fewer time. Smaller would also work here. Ratios, however, such as on http://speed.pypy.org/comparison/ cannot be 'more or less' or 'more or fewer'. They have to be greater (or larger) or smaller. Thus we need 'smaller' here. Laura From dcolish at gmail.com Mon Feb 28 18:51:22 2011 From: dcolish at gmail.com (Dan Colish) Date: Mon, 28 Feb 2011 09:51:22 -0800 Subject: [pypy-dev] _scproxy module and tweaks for osx Message-ID: <237B76A5746A4108B33E98626E152D15@gmail.com> I've been playing with a few ideas on OSX to get the build to be more stable. Attached is a patch that does this, but it is far from complete. There are still bugs in the way that frameworks are used when the final Makefile is created. In addition, there are problems with libintl that I'd like to be a little more pragmatic about. The _scproxy module itself works with the framework hardcoded fixes. Anyway, I'd appreciate some feedback and also so thoughts on how best to deal with various build environments on OSX. -- Dan -------------- next part -------------- A non-text attachment was scrubbed... Name: stable-scproxy.diff Type: application/octet-stream Size: 16448 bytes Desc: not available URL: