From tismer at stackless.com  Mon Nov  1 18:02:58 2010
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 01 Nov 2010 18:02:58 +0100
Subject: [pypy-dev]
 =?iso-8859-1?q?Next_sprint=3A_D=FCsseldorf=2C_end_of_O?=
 =?iso-8859-1?q?ctober?=
In-Reply-To: <AANLkTinX9u-TQR2B_8yV1Ynwb3sL=E4=TBGfBJ8nM1qr@mail.gmail.com>
References: <AANLkTinX9u-TQR2B_8yV1Ynwb3sL=E4=TBGfBJ8nM1qr@mail.gmail.com>
Message-ID: <4CCEF2C2.7040508@stackless.com>

On 10/1/10 11:31 AM, Armin Rigo wrote:
> Hi all!
>
> D?sseldorf PyPy sprint October 25th-31st 2010
> =====================================================
>
...

Hi Armin,

this became a trap for me.
The PyPy sprints were always listed in the sprint list as well.
But this time it was not, and I used the entry from 2009
without recognizing because of my reduced sight.
I even had answered to that, with no reaction, obviously.

That means the sprint is over, and I was about to start my travel
to D?sseldorf on Friday. :-(

Good that I talked to Stephan Diehl today, who made me check again.

I really wanted to come and get a deeper insight into PyPy's status.
Feeling pretty disappointed.

ciao -- chris

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key ->  http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/



From hakan at debian.org  Mon Nov  1 22:13:07 2010
From: hakan at debian.org (Hakan Ardo)
Date: Mon, 1 Nov 2010 22:13:07 +0100
Subject: [pypy-dev] pypy_g_action_dispatcher
Message-ID: <AANLkTi=EYF5XUHdyV_08iQ3-tDFdQrjFOzY9XA_U79zP@mail.gmail.com>

Hi,
after 1016420 iterations of the loop below, pypy_g_action_dispatcher()
is called for the first time. After that it is called every 100
iterations or so. Is this intentional?

import sys
for i in range(10000000):
    if i%20 == 0: sys.stderr.write('%d\n'%i)

-- 
H?kan Ard?


From arigo at tunes.org  Wed Nov  3 13:32:11 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Nov 2010 13:32:11 +0100
Subject: [pypy-dev] pypy_g_action_dispatcher
In-Reply-To: <AANLkTi=EYF5XUHdyV_08iQ3-tDFdQrjFOzY9XA_U79zP@mail.gmail.com>
References: <AANLkTi=EYF5XUHdyV_08iQ3-tDFdQrjFOzY9XA_U79zP@mail.gmail.com>
Message-ID: <AANLkTimWY+FcnWOHNKoLqicb-T0AuGbNsy65BOGcb7zB@mail.gmail.com>

Hi Hakan,

On Mon, Nov 1, 2010 at 10:13 PM, Hakan Ardo <hakan at debian.org> wrote:
> after 1016420 iterations of the loop below, pypy_g_action_dispatcher()
> is called for the first time. After that it is called every 100
> iterations or so. Is this intentional?

No, strange.

Armin


From cfbolz at gmx.de  Wed Nov  3 13:38:47 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 03 Nov 2010 13:38:47 +0100
Subject: [pypy-dev]
 =?utf-8?q?Next_sprint=3A_D=C3=BCsseldorf=2C_end_of_Oct?=
 =?utf-8?q?ober?=
In-Reply-To: <4CCEF2C2.7040508@stackless.com>
References: <AANLkTinX9u-TQR2B_8yV1Ynwb3sL=E4=TBGfBJ8nM1qr@mail.gmail.com>
	<4CCEF2C2.7040508@stackless.com>
Message-ID: <4CD157D7.1060701@gmx.de>

On 11/01/2010 06:02 PM, Christian Tismer wrote:
> On 10/1/10 11:31 AM, Armin Rigo wrote:
>> Hi all!
>>
>> D?sseldorf PyPy sprint October 25th-31st 2010
>> =====================================================
>>
> ...
>
> Hi Armin,
>
> this became a trap for me.
> The PyPy sprints were always listed in the sprint list as well.
> But this time it was not, and I used the entry from 2009
> without recognizing because of my reduced sight.
> I even had answered to that, with no reaction, obviously.
>
> That means the sprint is over, and I was about to start my travel
> to D?sseldorf on Friday. :-(
>
> Good that I talked to Stephan Diehl today, who made me check again.
>
> I really wanted to come and get a deeper insight into PyPy's status.
> Feeling pretty disappointed.

Hi Christian,

If you just want to find out what is currently going on with PyPy, you 
could just come at any time to D?sseldorf for some days. Armin and me 
are here all the time, and Anto is here until end of November. Just send 
us a mail (off-list, I guess) to discuss timing.

Cheers,

Carl Friedrich


From tismer at stackless.com  Wed Nov  3 22:33:13 2010
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 03 Nov 2010 22:33:13 +0100
Subject: [pypy-dev]
 =?utf-8?q?Next_sprint=3A_D=C3=BCsseldorf=2C_end_of_Oct?=
 =?utf-8?q?ober?=
In-Reply-To: <4CD157D7.1060701@gmx.de>
References: <AANLkTinX9u-TQR2B_8yV1Ynwb3sL=E4=TBGfBJ8nM1qr@mail.gmail.com>	<4CCEF2C2.7040508@stackless.com>
	<4CD157D7.1060701@gmx.de>
Message-ID: <4CD1D519.6060806@stackless.com>

On 11/3/10 1:38 PM, Carl Friedrich Bolz wrote:
> On 11/01/2010 06:02 PM, Christian Tismer wrote:
>> On 10/1/10 11:31 AM, Armin Rigo wrote:
>>> Hi all!
>>>
>>> D?sseldorf PyPy sprint October 25th-31st 2010
>>> =====================================================
>>>
>> ...
>>
>> Hi Armin,
>>
>> this became a trap for me.
>> The PyPy sprints were always listed in the sprint list as well.
>> But this time it was not, and I used the entry from 2009
>> without recognizing because of my reduced sight.
>> I even had answered to that, with no reaction, obviously.
>>
>> That means the sprint is over, and I was about to start my travel
>> to D?sseldorf on Friday. :-(
>>
>> Good that I talked to Stephan Diehl today, who made me check again.
>>
>> I really wanted to come and get a deeper insight into PyPy's status.
>> Feeling pretty disappointed.
> Hi Christian,
>
> If you just want to find out what is currently going on with PyPy, you
> could just come at any time to D?sseldorf for some days. Armin and me
> are here all the time, and Anto is here until end of November. Just send
> us a mail (off-list, I guess) to discuss timing.
>

Oh, that's a nice offer, thanks very much.
I will check my schedule and contact you
in private.

cheers - chris

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key ->  http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/



From anto.cuni at gmail.com  Fri Nov  5 17:30:02 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 05 Nov 2010 17:30:02 +0100
Subject: [pypy-dev] Migration to mercurial
Message-ID: <4CD4310A.5030201@gmail.com>

Hi all,

as some of you might have noticed by watching the IRC channel, we are finally
migrating to mercurial.  The actual conversion of the repository will happen
at some point during next week (or maybe the week after).

One issue that we have to care about is how to convert the author names: in
the current svn repository each author is simply indicated by its codespeak
username, while with mercurial this is no longer possible as there is no
longer a "centralized" server where everyone has an account.

In mercurial an author is just a string, but the standard way is put both the
real name and a valid email address to uniquely identify the author.  This is
exploited by websites like e.g. bitbucket to link the author of the commit to
the bitbucket user.

For doing the conversion, we will use the following usermap:
http://codespeak.net/svn/pypy/extradoc/planning/hg-migration/usermap.txt

By default, we map each username to its real name, as found in the /etc/passwd
file on codespeak.net.  Because of the reasons I explained above, I strongly
recommend everyone to put its email address there, so that it will be there in
the final mercurial repository.

If for any reason you don't want your real name to be in the mercurial
repository, please kill the corresponding line in the file.

Note that it won't be possible to change the info after the conversion has
been made.

ciao,
Anto


From holger at merlinux.eu  Wed Nov 10 16:57:30 2010
From: holger at merlinux.eu (holger krekel)
Date: Wed, 10 Nov 2010 16:57:30 +0100
Subject: [pypy-dev] PyPy has a non-profit foundation now
Message-ID: <20101110155729.GD1009@trillke.net>

Hi all,

i'd like to point you to your recent joining to
the Software Freedom Conservancy (SFC).  We decided
it's better to bite the bullet and go GPL in exchange
for working with a good non-profit foundation and getting
some donations. Any objections? 

Nah, just kidding.  But what is true is that the PyPy project 
joined the Software Freedom Conservancy, see our posts and theirs:

    http://morepypy.blogspot.com/2010/11/speeding-up-pypy-by-donations.html

    http://sfconservancy.org/news/2010/nov/10/pypy-joins/

Many thanks to Bradley M. Kuhn from the SFC for helping to implement
a smooth process so far.  And now let's see if some people find
some spare money so we can convene on how to spend it for
PyPy improvements.  Oh, and if you happen to know how
to personally approach the right guys in companies to 
send some donations the PyPy way, no need to be shy.

cheers,
holger


From fijall at gmail.com  Wed Nov 10 21:53:38 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 10 Nov 2010 22:53:38 +0200
Subject: [pypy-dev] Memory consumption of pypy-c translate.py
Message-ID: <AANLkTimh3CwQcA88eVwnPL1M2fX5fTPr-xGTi3hDpW4K@mail.gmail.com>

Hello.

I did a bit of measurments using various techniques and
pypy-c-64bit-jit and the outcome is that after a bit of annotation,
where memory consumption is 1200M, we get:

* 31M of mmaped code blocks for assembler
* about 150M I can account for that is jit resume data
* about 150M of the rest I can account for.

Where is everything else? Can GC trash some memory (like a lot).

Here is output of dump.py -r http://codespeak.net/~fijal/jit64.out


From arigo at tunes.org  Fri Nov 12 09:19:32 2010
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Nov 2010 09:19:32 +0100
Subject: [pypy-dev] Release 1.4
Message-ID: <AANLkTim2Z8wHL4quwBhSw2CERKWActBcDHdhjACg6yQA@mail.gmail.com>

Hi all,

We are preparing for the next release of PyPy.

A branch was already created
http://codespeak.net/svn/pypy/release/1.4.x, twice, but it may get
thrown away and re-created again.  I have 3 things that I would like
to fix before the release:

1. Mac OS/X 64.  We have there an issue with asmgcc not finding the gc roots.

2. Try to understand if there is a leak in pypy-c-jit or not.  The
issue is that it consumes quite a lot of memory and fijal could not,
so far, really figure out where it comes from.

3. Optionally, depending on 2, implement freeing old JIT-generated
assembler and supporting objects.

For 3, I have in mind the following lightweight plan: put an "age"
counter on each TreeLoop object, reset it to zero whenever we run the
corresponding code, and create an extension of the asmgcc root finder
that also resets to zero the counter of loops found in the stack.
After every GC collection we increment the age counter.  This should
ensure that the age of any live loop never goes past 1; on the other
hand, loops whose age reach some limit (like 20 or so) have not been
run for a long time, and are freed.  If we do it right it would free
all supporting data structures, and with a __del__ somewhere we could
also free the assembler itself (which may only account for 25% of ram
usage).


A bient?t,

Armin.


From phyo.arkarlwin at gmail.com  Fri Nov 12 21:18:22 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sat, 13 Nov 2010 02:48:22 +0630
Subject: [pypy-dev] Release 1.4
In-Reply-To: <AANLkTim2Z8wHL4quwBhSw2CERKWActBcDHdhjACg6yQA@mail.gmail.com>
References: <AANLkTim2Z8wHL4quwBhSw2CERKWActBcDHdhjACg6yQA@mail.gmail.com>
Message-ID: <AANLkTinb8hytVoPH6_ApRFTsEDC+BbhX95XhPzmbSNY9@mail.gmail.com>

Is python-mysql gonna work in 1.4 ? i cant wait to run my webserver via
pypy-jit

On Fri, Nov 12, 2010 at 2:49 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> We are preparing for the next release of PyPy.
>
> A branch was already created
> http://codespeak.net/svn/pypy/release/1.4.x, twice, but it may get
> thrown away and re-created again.  I have 3 things that I would like
> to fix before the release:
>
> 1. Mac OS/X 64.  We have there an issue with asmgcc not finding the gc
> roots.
>
> 2. Try to understand if there is a leak in pypy-c-jit or not.  The
> issue is that it consumes quite a lot of memory and fijal could not,
> so far, really figure out where it comes from.
>
> 3. Optionally, depending on 2, implement freeing old JIT-generated
> assembler and supporting objects.
>
> For 3, I have in mind the following lightweight plan: put an "age"
> counter on each TreeLoop object, reset it to zero whenever we run the
> corresponding code, and create an extension of the asmgcc root finder
> that also resets to zero the counter of loops found in the stack.
> After every GC collection we increment the age counter.  This should
> ensure that the age of any live loop never goes past 1; on the other
> hand, loops whose age reach some limit (like 20 or so) have not been
> run for a long time, and are freed.  If we do it right it would free
> all supporting data structures, and with a __del__ somewhere we could
> also free the assembler itself (which may only account for 25% of ram
> usage).
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101113/4fd68032/attachment.html>

From andrewfr_ice at yahoo.com  Sat Nov 13 15:32:06 2010
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Sat, 13 Nov 2010 06:32:06 -0800 (PST)
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
Message-ID: <627056.25013.qm@web120714.mail.ne1.yahoo.com>

Hi:

I finally got my hands on a machine big enough to compile pypy-c. I want to try pypy-c with the greenlet package. As I mentioned in previous posts, I want to see how the greenlet package blows up pypy-c and hope that it is a case of simply altering greenlets enough so it will work. 

I did a test run with -Ojit. It still took about 90 minutes to compile on a Mac with 4G of ram. That said, what would be the right switches to create a version can be placed in a debugger? Where does the C code go - or would this code even be useful. Any suggestions to speed up the process.

Cheers,
Andrew




From santagada at gmail.com  Sat Nov 13 15:39:11 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 13 Nov 2010 12:39:11 -0200
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
In-Reply-To: <627056.25013.qm@web120714.mail.ne1.yahoo.com>
References: <627056.25013.qm@web120714.mail.ne1.yahoo.com>
Message-ID: <AANLkTi=QkM6qdGx8CYakE_N12Ks_m_anp2sa_T-2LZhz@mail.gmail.com>

On Sat, Nov 13, 2010 at 12:32 PM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> Hi:
>
> I finally got my hands on a machine big enough to compile pypy-c. I want to try pypy-c with the greenlet package. As I mentioned in previous posts, I want to see how the greenlet package blows up pypy-c and hope that it is a case of simply altering greenlets enough so it will work.
>
> I did a test run with -Ojit. It still took about 90 minutes to compile on a Mac with 4G of ram. That said, what would be the right switches to create a version can be placed in a debugger? Where does the C code go - or would this code even be useful. Any suggestions to speed up the process.

One way to speed up is to use nightly builds
(http://buildbot.pypy.org/nightly/trunk/) :)

I still can't see how the greenlet for cpython could possibly work on
pypy. If you are talking about
http://codespeak.net/pypy/dist/pypy/doc/stackless.html#greenlets then
I think the "only" problem is that it doesn't work with the jit right?

-- 
Leonardo Santagada


From andrewfr_ice at yahoo.com  Sat Nov 13 18:40:39 2010
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Sat, 13 Nov 2010 09:40:39 -0800 (PST)
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
In-Reply-To: <AANLkTi=QkM6qdGx8CYakE_N12Ks_m_anp2sa_T-2LZhz@mail.gmail.com>
Message-ID: <639562.33217.qm@web120715.mail.ne1.yahoo.com>

Hi Leonardo:

--- On Sat, 11/13/10, Leonardo Santagada <santagada at gmail.com> wrote:

> From: Leonardo Santagada <santagada at gmail.com>
> Subject: Re: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless support)
> To: "Andrew Francis" <andrewfr_ice at yahoo.com>
> Cc: pypy-dev at codespeak.net
> Date: Saturday, November 13, 2010, 6:39 AM

> I still can't see how the greenlet for cpython could
> possibly work on pypy. If you are talking about
> http://codespeak.net/pypy/dist/pypy/doc/stackless.html#greenlets
> then I think the "only" problem is that it doesn't work with the
> jit right?

As I have stated in earlier posts, I am a newbie. However I want to help with Maciej Fijalkowski's suggestions about integrating Stackless with the JIT. I figured a way to start is to see what it would take to get greenlet style hard switching done. Since greenlet is a C extension, let's be naive, compile it and pypy-c, run it in a debugger and see exactly how it fails. Besides reading the documentation inside the greenlet.c source, I can take a similar approach with Standard C - that is run it in a debugger and see what happens. This way I get a feel for the issues and get familiar with the tools.

Maybe trying to work off the greenlet source will prove to be too difficult.

Right now, I have a computer that will compile pypy-c. However it takes a long time. I just want to make sure that everything is in place to carry out my plan. Also it would be nice if folks can spot problems in the approach. I know I need to learn more about stuff like shadow stacks. Also I have started to read documentation on the bytecode interpreter. 

Cheers,
Andrew







      


From santagada at gmail.com  Sat Nov 13 19:38:30 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 13 Nov 2010 16:38:30 -0200
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
In-Reply-To: <639562.33217.qm@web120715.mail.ne1.yahoo.com>
References: <AANLkTi=QkM6qdGx8CYakE_N12Ks_m_anp2sa_T-2LZhz@mail.gmail.com>
	<639562.33217.qm@web120715.mail.ne1.yahoo.com>
Message-ID: <AANLkTimOGgp1qzL89ZxzTnebjPCeU1SZcZJxtbLUODDY@mail.gmail.com>

On Sat, Nov 13, 2010 at 3:40 PM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
> --- On Sat, 11/13/10, Leonardo Santagada <santagada at gmail.com> wrote:
>> I still can't see how the greenlet for cpython could
>> possibly work on pypy. If you are talking about
>> http://codespeak.net/pypy/dist/pypy/doc/stackless.html#greenlets
>> then I think the "only" problem is that it doesn't work with the
>> jit right?
>
> As I have stated in earlier posts, I am a newbie. However I want to help with Maciej Fijalkowski's suggestions about integrating Stackless with the JIT. I figured a way to start is to see what it would take to get greenlet style hard switching done. Since greenlet is a C extension, let's be naive, compile it and pypy-c, run it in a debugger and see exactly how it fails. Besides reading the documentation inside the greenlet.c source, I can take a similar approach with Standard C - that is run it in a debugger and see what happens. This way I get a feel for the issues and get familiar with the tools.
>
> Maybe trying to work off the greenlet source will prove to be too difficult.

I will repeat what armin said, but I think you should reread what he
said in the previous thread, the gist is it will probably not work and
give you strange crashes and there is probably no way to fix them.

I think what fijal meant is to use the ideas in greenlets.c and map
them to pypy, but not directly trying to run the extension on top of
pypy.

Here is a list of what is crazy about the idea:
- greenlets is a c module, but the stuff it does with the stack is
completely outside the C API which is what pypy supports from cpython.
- It will (probably) break the gc. It will not find the roots (address
on the stack to memory on the heap) from functions that are taken out
of the stack
- The layout of the stack is probably not the same also and that will
also break.

> Right now, I have a computer that will compile pypy-c. However it takes a long time. I just want to make sure that everything is in place to carry out my plan. Also it would be nice if folks can spot problems in the approach. I know I need to learn more about stuff like shadow stacks. Also I have started to read documentation on the bytecode interpreter.

My idea to make things faster for you:

- Forget about running greenlets inside pypy-c after running and
making sure that armin prediction is true (that it will just blowup),
don't try to fix it. I also think that greenlets are more or less
independent from the bytecode interpreter, then don't waste your time
with that either.
- To translate faster and using less memory turn your python to 32bit
and use --no-allworkingmodules.
- Look at how greenlets.c work and talk to fijal and the other on irc
after/during you understand how it unwinds the stack. Write tests for
it (or reuse the ones that probably exist for stackless and
greenlets).
- Make a solid plan on how to do it.
- Have fun

:)

ps: I hope I was clear (and also correct)
-- 
Leonardo Santagada


From andrewfr_ice at yahoo.com  Sun Nov 14 16:11:32 2010
From: andrewfr_ice at yahoo.com (Andrew Francis)
Date: Sun, 14 Nov 2010 07:11:32 -0800 (PST)
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
Message-ID: <64943.47473.qm@web120703.mail.ne1.yahoo.com>

Hello Leonardo:

--- On Sat, 11/13/10, Leonardo Santagada <santagada at gmail.com> wrote:

> From: Leonardo Santagada <santagada at gmail.com>
> Subject: Re: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless support)
> To: "Andrew Francis" <andrewfr_ice at yahoo.com>
> Cc: pypy-dev at codespeak.net
> Date: Saturday, November 13, 2010, 10:38 AM
> On Sat, Nov 13, 2010 at 3:40 PM,
> Andrew Francis <andrewfr_ice at yahoo.com>
> wrote:
> > --- On Sat, 11/13/10, Leonardo Santagada <santagada at gmail.com>
> wrote:

> I think what fijal meant is to use the ideas in greenlets.c
> and map them to pypy, but not directly trying to run the extension
> on top of pypy.

I understood what Fijal said. I thought why not try modifying it?

> Here is a list of what is crazy about the idea:
> - greenlets is a c module, but the stuff it does with the
> stack is completely outside the C API which is what pypy supports
> from cpython.

Leonardo, I don't want to sound argumentative but conceptually does this differ from what greenlets does with Standard Python? If I am not mistaken, greenlets (like Stackless Python) at some point need to do some very machine dependent low-level stuff to move  stack frames to and from a heap. That is not in the C API either. 

> - It will (probably) break the gc. It will not find the
> roots (address on the stack to memory on the heap) from functions that >are taken out of the stack - The layout of the stack is probably not the >same also and that will also break.

I acknowledge all of these concerns. However what I would like to figure out is if the greenlet.c code is sufficiently modular so these parts can be isolated. I figure it would be good strategy to try to keep most of greenlet's high level 

Maybe folks can take a quick look at greenlet.c and off-hand state what would change. I would figure that low level routines (i.e., like g_save() and slp_restore_state(). In this fashion we have more facts to see if this approach should be nipped in the bud. 

> - To translate faster and using less memory turn your
> python to 32bit and use --no-allworkingmodules.

Great! I'll try this.

> - Look at how greenlets.c work and talk

I am doing this as we speak. I'll make IRC appearances once I have really has something new to add. 

> ps: I hope I was clear (and also correct)

I thank you for your input. Right now, I need more information and to learn more. 

Cheers,
Andrew





      


From arigo at tunes.org  Mon Nov 15 13:07:57 2010
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 15 Nov 2010 13:07:57 +0100
Subject: [pypy-dev] Advice on Compiling Pypy-c for debugging (Stackless
	support)
In-Reply-To: <64943.47473.qm@web120703.mail.ne1.yahoo.com>
References: <64943.47473.qm@web120703.mail.ne1.yahoo.com>
Message-ID: <AANLkTikPAYChcXfWhyDktdbomO9RSyj3onMubvxVVYky@mail.gmail.com>

Hi Andrew,

On Sun, Nov 14, 2010 at 4:11 PM, Andrew Francis <andrewfr_ice at yahoo.com> wrote:
>> I think what fijal meant is to use the ideas in greenlets.c
>> and map them to pypy, but not directly trying to run the extension
>> on top of pypy.
>
> I understood what Fijal said. I thought why not try modifying it?

Because it will not work at all.  This approach is completely doomed.

And that's probably the last time you'll get an answer to the same
question, so I've tried to be as clear as I possibly can :-)


Armin


From drsalists at gmail.com  Tue Nov 16 01:25:46 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 15 Nov 2010 16:25:46 -0800
Subject: [pypy-dev] gdbm
Message-ID: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>

I've put together a gdbm wrapper module using ctypes, and tested against
pypy with the included test_gdbm.  Is there interest in adding it to pypy?

It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone wants
to look it over.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101115/3c341da2/attachment.html>

From amauryfa at gmail.com  Tue Nov 16 02:22:56 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 16 Nov 2010 02:22:56 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
Message-ID: <AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>

Hi,

2010/11/16 Dan Stromberg <drsalists at gmail.com>:
> I've put together a gdbm wrapper module using ctypes, and tested against
> pypy with the included test_gdbm.? Is there interest in adding it to pypy?
>
> It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone wants
> to look it over.

I don't know exactly what gdbm is, but it looks very similar to dbm.py
already in pypy:
http://codespeak.net/svn/pypy/trunk/lib_pypy/dbm.py

-- 
Amaury Forgeot d'Arc


From drsalists at gmail.com  Tue Nov 16 03:17:06 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 15 Nov 2010 18:17:06 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
Message-ID: <AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>

Yes, the dbm module in pypy is basically like the bsddb module in cpython.

cpython includes modules for bsddb, gdbm, and more.

I tend to prefer gdbm over bsddb, because I've seen bsddb databases get
corrupt too many times - EG, when a filesystem overflows.  bsddb might be a
little faster though; I've never compared their performance.

On Mon, Nov 15, 2010 at 5:22 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> Hi,
>
> 2010/11/16 Dan Stromberg <drsalists at gmail.com>:
> > I've put together a gdbm wrapper module using ctypes, and tested against
> > pypy with the included test_gdbm.  Is there interest in adding it to
> pypy?
> >
> > It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone
> wants
> > to look it over.
>
> I don't know exactly what gdbm is, but it looks very similar to dbm.py
> already in pypy:
> http://codespeak.net/svn/pypy/trunk/lib_pypy/dbm.py
>
> --
> Amaury Forgeot d'Arc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101115/42491b09/attachment.html>

From lac at openend.se  Tue Nov 16 03:22:53 2010
From: lac at openend.se (Laura Creighton)
Date: Tue, 16 Nov 2010 03:22:53 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: Message from "Amaury Forgeot d'Arc" <amauryfa@gmail.com> of "Tue,
	16 Nov 2010 02:22:56 +0100."
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com> 
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com> 
Message-ID: <201011160222.oAG2Mrfv004177@theraft.openend.se>

In a message of Tue, 16 Nov 2010 02:22:56 +0100, "Amaury Forgeot d'Arc" writes:
>Hi,
>
>2010/11/16 Dan Stromberg <drsalists at gmail.com>:
>> I've put together a gdbm wrapper module using ctypes, and tested agains
>t
>> pypy with the included test_gdbm.? Is there interest in adding it to pyp
>y?
>>
>> It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone 
>wants
>> to look it over.
>
>I don't know exactly what gdbm is, but it looks very similar to dbm.py
>already in pypy:
>http://codespeak.net/svn/pypy/trunk/lib_pypy/dbm.py

Both gdbm and dbm are part of the python standard library, so I'd guess we'd
better have both of them.

http://docs.python.org/library/gdbm.html 

Laura

>Amaury Forgeot d'Arc



From drsalists at gmail.com  Tue Nov 16 04:30:11 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 15 Nov 2010 19:30:11 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <201011160222.oAG2Mrfv004177@theraft.openend.se>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<amauryfa@gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<201011160222.oAG2Mrfv004177@theraft.openend.se>
Message-ID: <AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>

On Mon, Nov 15, 2010 at 6:22 PM, Laura Creighton <lac at openend.se> wrote:

> In a message of Tue, 16 Nov 2010 02:22:56 +0100, "Amaury Forgeot d'Arc"
> writes:
> >Hi,
> >
> >2010/11/16 Dan Stromberg <drsalists at gmail.com>:
> >> I've put together a gdbm wrapper module using ctypes, and tested agains
> >t
> >> pypy with the included test_gdbm.  Is there interest in adding it to pyp
> >y?
> >>
> >> It's at http://stromberg.dnsalias.org/svn/gdbm-ctypes/trunk/ if anyone
> >wants
> >> to look it over.
> >
> >I don't know exactly what gdbm is, but it looks very similar to dbm.py
> >already in pypy:
> >http://codespeak.net/svn/pypy/trunk/lib_pypy/dbm.py
>
> Both gdbm and dbm are part of the python standard library, so I'd guess
> we'd
> better have both of them.
>
> http://docs.python.org/library/gdbm.html
>
> Laura
>
> Sounds good to me :)

BTW, it might cause confusion down the road to call something that is
basically like cpython's bsddb (Berkeley DB) by the name "dbm" in pypy's
library.  In the cpython standard library, "dbm" is an interface to ndbm
databases.  These all provide the same dictionary-like interface to Python
programs, but have somewhat different API's to C, and pretty different,
incompatible on-disk representations.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101115/0cd72bfa/attachment.html>

From anto.cuni at gmail.com  Tue Nov 16 13:17:34 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 16 Nov 2010 13:17:34 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
Message-ID: <4CE2765E.2020507@gmail.com>

Hi Dan,
first: thanks for your help :-)

On 16/11/10 03:17, Dan Stromberg wrote:
>
> Yes, the dbm module in pypy is basically like the bsddb module in cpython.
>
> cpython includes modules for bsddb, gdbm, and more.
>
> I tend to prefer gdbm over bsddb, because I've seen bsddb databases get
> corrupt too many times - EG, when a filesystem overflows.  bsddb might be a
> little faster though; I've never compared their performance.

So, if I understand correctly you are saying that we should rename our dbm.py 
to bsdb.py, and write a new dbm.py which can use either bsdb or gdbm?
Sounds fine, do you feel like implementing it? :-)

Moreover, I also agree with amaury that your code is very similar to the one 
in the current dbm.py, so we should maybe try to refactor things to share 
common parts between the twos.

ciao,
Anto


From drsalists at gmail.com  Tue Nov 16 17:27:03 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 16 Nov 2010 08:27:03 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CE2765E.2020507@gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
Message-ID: <AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>

On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

>
> On 16/11/10 03:17, Dan Stromberg wrote:
>
>>
>> Yes, the dbm module in pypy is basically like the bsddb module in cpython.
>>
>> cpython includes modules for bsddb, gdbm, and more.
>>
>> I tend to prefer gdbm over bsddb, because I've seen bsddb databases get
>> corrupt too many times - EG, when a filesystem overflows.  bsddb might be
>> a
>> little faster though; I've never compared their performance.
>>
>
>
> So, if I understand correctly you are saying that we should rename our
> dbm.py to bsdb.py, and write a new dbm.py which can use either bsdb or gdbm?
>
I think it's anydbm that can use whatever among dbm, bsddb, gdbm and
dumbdbm, as it sees fit.  TTBoMK, it's not until python 3.x that dbm becomes
a sort of unifying module hierarchy.


> Sounds fine, do you feel like implementing it? :-)
>
> Moreover, I also agree with amaury that your code is very similar to the
> one in the current dbm.py, so we should maybe try to refactor things to
> share common parts between the twos.
>
I'd be happy to.

Would sharing based on inheritance or a more functional approach be
preferred?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101116/197ef989/attachment.html>

From arigo at tunes.org  Tue Nov 16 17:47:35 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 16 Nov 2010 17:47:35 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
Message-ID: <AANLkTikDn_su7H_TMtUFeW7kmgoBySsQ+qYmMSFWtu7h@mail.gmail.com>

Hi,

On Tue, Nov 16, 2010 at 5:27 PM, Dan Stromberg <drsalists at gmail.com> wrote:
>> So, if I understand correctly you are saying that we should rename our
>> dbm.py to bsdb.py, and write a new dbm.py which can use either bsdb or gdbm?
>
> I think it's anydbm that can use whatever among dbm, bsddb, gdbm and
> dumbdbm, as it sees fit.? TTBoMK, it's not until python 3.x that dbm becomes
> a sort of unifying module hierarchy.

Yes, in Python 2.x, dbm.py is very specifically an interface to the
Unix dbm library (see e.g. man dbm_open).  At the level of C, the gdbm
interface is some kind of extension of that.  It's not related to
bsddb, which has a very different interface.

> Would sharing based on inheritance or a more functional approach be
> preferred?

I think either is fine (or even not sharing at all if it turns out to
be too much of a mess in practice).


A bient?t,

Armin.


From exarkun at twistedmatrix.com  Tue Nov 16 17:38:25 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Tue, 16 Nov 2010 16:38:25 -0000
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
Message-ID: <20101116163825.2040.1287738136.divmod.xquotient.932@localhost.localdomain>

On 04:27 pm, drsalists at gmail.com wrote:
>On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com> 
>wrote:
>>Sounds fine, do you feel like implementing it? :-)
>>
>>Moreover, I also agree with amaury that your code is very similar to 
>>the
>>one in the current dbm.py, so we should maybe try to refactor things 
>>to
>>share common parts between the twos.
>I'd be happy to.
>
>Would sharing based on inheritance or a more functional approach be
>preferred?

No, avoid inheritance where possible.  Composition is preferred.

As for "functional", I don't really know what approach you're describing 
with that word.

Jean-Paul


From arigo at tunes.org  Tue Nov 16 19:12:57 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 16 Nov 2010 19:12:57 +0100
Subject: [pypy-dev] Release 1.4
In-Reply-To: <AANLkTinb8hytVoPH6_ApRFTsEDC+BbhX95XhPzmbSNY9@mail.gmail.com>
References: <AANLkTim2Z8wHL4quwBhSw2CERKWActBcDHdhjACg6yQA@mail.gmail.com>
	<AANLkTinb8hytVoPH6_ApRFTsEDC+BbhX95XhPzmbSNY9@mail.gmail.com>
Message-ID: <AANLkTi=mK1A87d1d8UPr1u9JARxQ4vZAKURE8WbKGe_i@mail.gmail.com>

Hi Phyo,

On Fri, Nov 12, 2010 at 9:18 PM, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
> Is python-mysql gonna work in 1.4 ? i cant wait to run my webserver via
> pypy-jit

Yes, you can use the CPython C extension module.  It's a feature that
already worked more or less since PyPy 1.3.  See explanations in
http://morepypy.blogspot.com/2010/06/pypy-13-released.html .  Note
that _mysql.c first needs a small patch in order to work, which is
here: http://codespeak.net/svn/pypy/trunk/pypy/module/cpyext/patches/mysqldb.patch
.


Armin


From drsalists at gmail.com  Wed Nov 17 00:12:19 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 16 Nov 2010 15:12:19 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTikDn_su7H_TMtUFeW7kmgoBySsQ+qYmMSFWtu7h@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
	<AANLkTikDn_su7H_TMtUFeW7kmgoBySsQ+qYmMSFWtu7h@mail.gmail.com>
Message-ID: <AANLkTimW7rx+w=3pOPCy-oy-j4gOTX4phJ7xcdHeezeF@mail.gmail.com>

On Tue, Nov 16, 2010 at 8:47 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi,
>
> On Tue, Nov 16, 2010 at 5:27 PM, Dan Stromberg <drsalists at gmail.com>
> wrote:
> >> So, if I understand correctly you are saying that we should rename our
> >> dbm.py to bsdb.py, and write a new dbm.py which can use either bsdb or
> gdbm?
> >
> > I think it's anydbm that can use whatever among dbm, bsddb, gdbm and
> > dumbdbm, as it sees fit.  TTBoMK, it's not until python 3.x that dbm
> becomes
> > a sort of unifying module hierarchy.
>
> Yes, in Python 2.x, dbm.py is very specifically an interface to the
> Unix dbm library (see e.g. man dbm_open).  At the level of C, the gdbm
> interface is some kind of extension of that.  It's not related to
> bsddb, which has a very different interface.
>

Well, there's related, and then there's related.  bsddb provides a bunch of
operations, including a basic hash facility.  The API is not intended to be
the same as gdbm or ndbm (gdbm has a native interface and an ndbm
compatability interface), but the concept is similar between the 3 for one
part of bsddb - the part the cpython uses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101116/92a17838/attachment.html>

From drsalists at gmail.com  Wed Nov 17 00:24:58 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 16 Nov 2010 15:24:58 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <20101116163825.2040.1287738136.divmod.xquotient.932@localhost.localdomain>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
	<20101116163825.2040.1287738136.divmod.xquotient.932@localhost.localdomain>
Message-ID: <AANLkTi=QiNnmg0iqDE5AarMXgGE5VMFSQchntNu9-36v@mail.gmail.com>

On Tue, Nov 16, 2010 at 8:38 AM, <exarkun at twistedmatrix.com> wrote:

> On 04:27 pm, drsalists at gmail.com wrote:
> >On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com>
> >wrote:
> >>Sounds fine, do you feel like implementing it? :-)
> >>
> >>Moreover, I also agree with amaury that your code is very similar to
> >>the
> >>one in the current dbm.py, so we should maybe try to refactor things
> >>to
> >>share common parts between the twos.
> >I'd be happy to.
> >
> >Would sharing based on inheritance or a more functional approach be
> >preferred?
>
> No, avoid inheritance where possible.  Composition is preferred.
>
Due to performance?  Or "flat is better than nested" - as more of a
philosophical issue?  I'm not arguing for inheritance, just wanting to
understand the reasoning behind it.


> As for "functional", I don't really know what approach you're describing
> with that word.
>
I mean something like what you'd do in Lisp, ML or Haskell (not that I know
any of those that well).  I suppose in this case it would mean
functools.partial() as a decorator, or perhaps a wrapper around
functools.partial() used as a decorator.

Then again, maybe functools.partial() imposes its own performance penalty?
That's part of why I'm wondering if avoiding inheritance is a
performance-related goal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101116/75bf0635/attachment.html>

From p.giarrusso at gmail.com  Wed Nov 17 03:35:22 2010
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Wed, 17 Nov 2010 03:35:22 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=QiNnmg0iqDE5AarMXgGE5VMFSQchntNu9-36v@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
	<20101116163825.2040.1287738136.divmod.xquotient.932@localhost.localdomain>
	<AANLkTi=QiNnmg0iqDE5AarMXgGE5VMFSQchntNu9-36v@mail.gmail.com>
Message-ID: <AANLkTikX+4F1pMOxBYpaXSC1RFeVhkVH_eEEfLJBoLYo@mail.gmail.com>

On Wed, Nov 17, 2010 at 00:24, Dan Stromberg <drsalists at gmail.com> wrote:
>
> On Tue, Nov 16, 2010 at 8:38 AM, <exarkun at twistedmatrix.com> wrote:
>>
>> On 04:27 pm, drsalists at gmail.com wrote:
>> >On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com>
>> >wrote:
>> >>Sounds fine, do you feel like implementing it? :-)
>> >>
>> >>Moreover, I also agree with amaury that your code is very similar to
>> >>the
>> >>one in the current dbm.py, so we should maybe try to refactor things
>> >>to
>> >>share common parts between the twos.
>> >I'd be happy to.
>> >
>> >Would sharing based on inheritance or a more functional approach be
>> >preferred?

>> No, avoid inheritance where possible. ?Composition is preferred.

> Due to performance?? Or "flat is better than nested" - as more of a
> philosophical issue?? I'm not arguing for inheritance, just wanting to
> understand the reasoning behind it.

I answer on this because it is a general topic.
Preferring composition over inheritance is a general design principle,
not intended to improve performance; of course, there are cases where
inheritance is good, so it needs to be further qualified; but in a
very simple way, inheritance is often used where it is not
appropriate.

One of the reasons which come to my mind is that inheritance implies
closer coupling between subclasses and superclasses. If you look for
"composition vs inheritance" on Google, you will find quite a few
articles. One which looks good is this one:
http://www.artima.com/designtechniques/compoinh4.html
(you can also browse the whole article, I link to page 4 because it's
more interesting).

That article also points out that you might get some performance
overhead with composition instead of inheritance, which is the
opposite of what you said, and that makes some sense; however, in many
cases the overhead can be removed by an optimizer through inlining,
and that's one reason not to worry. The other reason not to worry is
that the overhead is anyway minor and thus design issues are more
important - doing otherwise would be premature optimization.

If you have a lot of experience with hard data which taught you that
you should avoid something upfront, and that data is still valid, then
you can avoid something upfront - and that's not the case here,
because when you change your target implementation, you'll need to
recheck about how it optimizes, and that takes long.

>> As for "functional", I don't really know what approach you're describing
>> with that word.

> I mean something like what you'd do in Lisp, ML or Haskell (not that I know
> any of those that well).? I suppose in this case it would mean
> functools.partial() as a decorator, or perhaps a wrapper around
> functools.partial() used as a decorator.

> Then again, maybe functools.partial() imposes its own performance penalty?
> That's part of why I'm wondering if avoiding inheritance is a
> performance-related goal.

I can't comment in detail on this, because I just had a look at the
code - but performance shouldn't be the main goal.

Anyway, it looks like most differences are about calling clearerr or
not, and about the module name in strings; probably one could abstract
that away through some other technique.

Bye!
-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/


From phyo.arkarlwin at gmail.com  Wed Nov 17 06:06:17 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Wed, 17 Nov 2010 11:36:17 +0630
Subject: [pypy-dev] Release 1.4
In-Reply-To: <AANLkTi=mK1A87d1d8UPr1u9JARxQ4vZAKURE8WbKGe_i@mail.gmail.com>
References: <AANLkTim2Z8wHL4quwBhSw2CERKWActBcDHdhjACg6yQA@mail.gmail.com>
	<AANLkTinb8hytVoPH6_ApRFTsEDC+BbhX95XhPzmbSNY9@mail.gmail.com>
	<AANLkTi=mK1A87d1d8UPr1u9JARxQ4vZAKURE8WbKGe_i@mail.gmail.com>
Message-ID: <AANLkTi=SkK4yXR5pD-_=J5dFkmpXDRLVmLpU7hxaFz_s@mail.gmail.com>

Oh thats cool !

THanks a lot i will try!

On 11/17/10, Armin Rigo <arigo at tunes.org> wrote:
> Hi Phyo,
>
> On Fri, Nov 12, 2010 at 9:18 PM, Phyo Arkar <phyo.arkarlwin at gmail.com>
> wrote:
>> Is python-mysql gonna work in 1.4 ? i cant wait to run my webserver via
>> pypy-jit
>
> Yes, you can use the CPython C extension module.  It's a feature that
> already worked more or less since PyPy 1.3.  See explanations in
> http://morepypy.blogspot.com/2010/06/pypy-13-released.html .  Note
> that _mysql.c first needs a small patch in order to work, which is
> here:
> http://codespeak.net/svn/pypy/trunk/pypy/module/cpyext/patches/mysqldb.patch
> .
>
>
> Armin
>


From fijall at gmail.com  Wed Nov 17 07:32:54 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 17 Nov 2010 08:32:54 +0200
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=QiNnmg0iqDE5AarMXgGE5VMFSQchntNu9-36v@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=SNsXfNAHZfFTP1Pk3LOrYzsm1LFAE6kcCbm40@mail.gmail.com>
	<20101116163825.2040.1287738136.divmod.xquotient.932@localhost.localdomain>
	<AANLkTi=QiNnmg0iqDE5AarMXgGE5VMFSQchntNu9-36v@mail.gmail.com>
Message-ID: <AANLkTimpV5V8udYDK0xDRzy8ZtvK9ow1LQBG849uaG-0@mail.gmail.com>

On Wed, Nov 17, 2010 at 1:24 AM, Dan Stromberg <drsalists at gmail.com> wrote:
>
> On Tue, Nov 16, 2010 at 8:38 AM, <exarkun at twistedmatrix.com> wrote:
>>
>> On 04:27 pm, drsalists at gmail.com wrote:
>> >On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com>
>> >wrote:
>> >>Sounds fine, do you feel like implementing it? :-)
>> >>
>> >>Moreover, I also agree with amaury that your code is very similar to
>> >>the
>> >>one in the current dbm.py, so we should maybe try to refactor things
>> >>to
>> >>share common parts between the twos.
>> >I'd be happy to.
>> >
>> >Would sharing based on inheritance or a more functional approach be
>> >preferred?
>>
>> No, avoid inheritance where possible. ?Composition is preferred.
>
> Due to performance?? Or "flat is better than nested" - as more of a
> philosophical issue?? I'm not arguing for inheritance, just wanting to
> understand the reasoning behind it.

I think one of the points is that those are originally not related in
CPython. And someone somewhere *will* use this and complain. People do
crazy stuff with Python.

>
>>
>> As for "functional", I don't really know what approach you're describing
>> with that word.
>
> I mean something like what you'd do in Lisp, ML or Haskell (not that I know
> any of those that well).? I suppose in this case it would mean
> functools.partial() as a decorator, or perhaps a wrapper around
> functools.partial() used as a decorator.
>
> Then again, maybe functools.partial() imposes its own performance penalty?
> That's part of why I'm wondering if avoiding inheritance is a
> performance-related goal.
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From arigo at tunes.org  Wed Nov 17 13:53:27 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 17 Nov 2010 13:53:27 +0100
Subject: [pypy-dev] Memory consumption of pypy-c translate.py
In-Reply-To: <AANLkTimh3CwQcA88eVwnPL1M2fX5fTPr-xGTi3hDpW4K@mail.gmail.com>
References: <AANLkTimh3CwQcA88eVwnPL1M2fX5fTPr-xGTi3hDpW4K@mail.gmail.com>
Message-ID: <AANLkTin9=nNdG2aG9u0HJLqRQjRg6A07b_1URxmRL3hL@mail.gmail.com>

Hi,

(Answering here in addition to on IRC, for reference.)

On Wed, Nov 10, 2010 at 9:53 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> * 31M of mmaped code blocks for assembler
> * about 150M I can account for that is jit resume data
> * about 150M of the rest I can account for.
>
> Where is everything else? Can GC trash some memory (like a lot).

Everything else is memory that has been freed, but not returned to the
OS.  (You get the same effect with CPython 2.5.) That's a bit
unexpected, because we call the C-level free() on it, but it is still
not returned to the OS; it seems to be caused by only a few mallocs()
that occur from time to time and stay alive for long (e.g. a few 4KB
pages needed by the GC for its various AddressStacks).  I verified it
in a custom C program that does the same calls to malloc() and free().
 If at the end we force free() on the remaining few 4KB blocks, then
the total memory use suddenly drops from (in this example) more than
200MB to almost 0.

This seems to be the main difference between the way pypy-c and
CPython 2.6 call the C-level malloc() and free().  Unsure how to fix
it.  (If the same program then asks for more memory again, it will
reuse this memory -- the libc malloc() at least ensures this, of
course.)


A bient?t,

Armin.


From renesd at gmail.com  Wed Nov 17 14:47:22 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Wed, 17 Nov 2010 13:47:22 +0000
Subject: [pypy-dev] Memory consumption of pypy-c translate.py
In-Reply-To: <AANLkTin9=nNdG2aG9u0HJLqRQjRg6A07b_1URxmRL3hL@mail.gmail.com>
References: <AANLkTimh3CwQcA88eVwnPL1M2fX5fTPr-xGTi3hDpW4K@mail.gmail.com>
	<AANLkTin9=nNdG2aG9u0HJLqRQjRg6A07b_1URxmRL3hL@mail.gmail.com>
Message-ID: <AANLkTinhK1Ha-OpxMvj9=HCsFgem_JWij9-FK5LVNAyV@mail.gmail.com>

Check out the brk syscall.

malloc is allocating more memory to itself.

You really need a custom malloc to change this behaviour.

cheers,

On Wed, Nov 17, 2010 at 12:53 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi,
>
> (Answering here in addition to on IRC, for reference.)
>
> On Wed, Nov 10, 2010 at 9:53 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> * 31M of mmaped code blocks for assembler
>> * about 150M I can account for that is jit resume data
>> * about 150M of the rest I can account for.
>>
>> Where is everything else? Can GC trash some memory (like a lot).
>
> Everything else is memory that has been freed, but not returned to the
> OS. ?(You get the same effect with CPython 2.5.) That's a bit
> unexpected, because we call the C-level free() on it, but it is still
> not returned to the OS; it seems to be caused by only a few mallocs()
> that occur from time to time and stay alive for long (e.g. a few 4KB
> pages needed by the GC for its various AddressStacks). ?I verified it
> in a custom C program that does the same calls to malloc() and free().
> ?If at the end we force free() on the remaining few 4KB blocks, then
> the total memory use suddenly drops from (in this example) more than
> 200MB to almost 0.
>
> This seems to be the main difference between the way pypy-c and
> CPython 2.6 call the C-level malloc() and free(). ?Unsure how to fix
> it. ?(If the same program then asks for more memory again, it will
> reuse this memory -- the libc malloc() at least ensures this, of
> course.)
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From drsalists at gmail.com  Thu Nov 18 01:17:09 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 17 Nov 2010 16:17:09 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CE2765E.2020507@gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
Message-ID: <AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>

On Tue, Nov 16, 2010 at 4:17 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> Hi Dan,
> first: thanks for your help :-)
>
>
> On 16/11/10 03:17, Dan Stromberg wrote:
>
>>
>> Yes, the dbm module in pypy is basically like the bsddb module in cpython.
>>
>> cpython includes modules for bsddb, gdbm, and more.
>>
>> I tend to prefer gdbm over bsddb, because I've seen bsddb databases get
>> corrupt too many times - EG, when a filesystem overflows.  bsddb might be
>> a
>> little faster though; I've never compared their performance.
>>
>
> So, if I understand correctly you are saying that we should rename our
> dbm.py to bsdb.py, and write a new dbm.py which can use either bsdb or gdbm?
> Sounds fine, do you feel like implementing it? :-)
>
> Moreover, I also agree with amaury that your code is very similar to the
> one in the current dbm.py, so we should maybe try to refactor things to
> share common parts between the twos.
>
> ciao,
> Anto
>

Wow, CPython's Berkeley DB interface is actually quite a bit more
comprehensive and complex than I'd realized.  This isn't just a matter of
renaming dbm.py to bsddb.py and refactoring a bit.  It's more of a time
commitment to something I don't use than I'd thought.

Althought pypy's current dbm.py implements something similar to cpython's
Berkeley DB interface, it isn't all that similar.  It uses a subset of the
same on-disk representations, but the API appears to be pretty different.
This is based on playing around in the unit tests and bsddb module a bit.

I actually suggest:
1) svn rename'ing dbm.py into some unused directory for history's sake; it
implements the ndbm _interface_ (a little oddly called "dbm" in cpython -
but I believe true "dbm" is "one database per program") well, but it's not
really all that similar to bsddb.
2) Adding the gdbm.py module I wrote, more or less verbatim.

I got into the project of merging these two things thinking that bsddb was
mostly just like the gdbm module, but bsddb is actually quite a bit more
involved, and is something I've pretty much stopped using due bad
experiences with bsddb database corruption from both cpython and C.

Given #1 and #2 above, anydbm should continue working, due to the presence
of gdbm and dumbdbm.

I guess I think that if someone has a need for bsddb (and it's assorted
interfaces), they probably should work on that.

Sound reasonable?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101117/0122a5b7/attachment.html>

From anto.cuni at gmail.com  Thu Nov 18 16:06:43 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 18 Nov 2010 16:06:43 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>	<4CE2765E.2020507@gmail.com>
	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
Message-ID: <4CE54103.6060603@gmail.com>

On 18/11/10 01:17, Dan Stromberg wrote:

Hi Dan,

[cut]
> Given #1 and #2 above, anydbm should continue working, due to the presence of
> gdbm and dumbdbm.
>
> I guess I think that if someone has a need for bsddb (and it's assorted
> interfaces), they probably should work on that.
>
> Sound reasonable?

Yes. To me it sounds reasonable. I don't know if it's correct because I never 
used any of those modules, but it's surely reasoable :-).

I think that the best would be if you get an account on codespeak and do that 
in a branch: this way we can follow your commits and the if everything is fine 
we merge it.

I personally cannot create new accounts on codespeak, so you should ask 
someone else for this (e.g. Armin Rigo): the easiest way is if you just come 
on #pypy on freenode.net and talk with us "real-time".

ciao,
Anto


From romcheg.prihod at gmail.com  Fri Nov 19 20:46:22 2010
From: romcheg.prihod at gmail.com (Roman Prykhodchenko)
Date: Fri, 19 Nov 2010 21:46:22 +0200
Subject: [pypy-dev] Memory error when importing optparse module
Message-ID: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>

Hi guys,


I'm trying to port my scripts to pypy but when I import the optparse module I get the error:

>>>> import optparse
Traceback (most recent call last):
 File "<console>", line 1, in <module>
 File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
   from gettext import gettext
 File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
   import locale, copy, os, re, struct, sys
 File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
   from _locale import *
 File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
   from ctypes import (Structure, POINTER, create_string_buffer,
 File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
   _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
 File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
   ffiargs, ffires, self._flags_)
MemoryError


OS: Mac OS Snow Leopard 10.6.5 

I tried to sign up your bugtracker to report a bug but the registration is broken  -- I got  the error message when I was trying to confirm my registration.
My login is prykhodchenko


---
Roman Prykhodchenko

From santagada at gmail.com  Sat Nov 20 05:09:00 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 20 Nov 2010 02:09:00 -0200
Subject: [pypy-dev] Needs some guidance in writing the kqueue part of select
Message-ID: <AANLkTinLJx3w1ksF2PorkJA+r=fMq92hMKEaMQmogkDq@mail.gmail.com>

I'm writing the kqueue part of the select module and would like some
guidance in how to fix some pieces of it. The code is here:
http://paste.pocoo.org/show/293812/ and the specific problems I'm
having are:

1) There is no UINTPTR_T in rffi, but INTPTR_T = SSIZE_T so can
UINTPTR_T be an alias to SIZE_T?

2) How do I declare that an argument is const in rffi?

But if anyone wants to take a look (or maybe more tests
http://paste.pocoo.org/show/293811/) please do. I just tested kqueue,
but now I need to test the kevent call, I'm planing on using
sockect_pair or a simple socket from rlib (if there is no socket_pair
there),

-- 
Leonardo Santagada


From fijall at gmail.com  Sat Nov 20 09:09:22 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 20 Nov 2010 10:09:22 +0200
Subject: [pypy-dev] Memory error when importing optparse module
In-Reply-To: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
References: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
Message-ID: <AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>

Hey.

Unfortunately we don't have a full-time OS X developer and it makes it
harder for this platform to be fully spported. Is your python 32bit or
64bit? How did you build that executable?

On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko
<romcheg.prihod at gmail.com> wrote:
> Hi guys,
>
>
> I'm trying to port my scripts to pypy but when I import the optparse module I get the error:
>
>>>>> import optparse
> Traceback (most recent call last):
> ?File "<console>", line 1, in <module>
> ?File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
> ? from gettext import gettext
> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
> ? import locale, copy, os, re, struct, sys
> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
> ? from _locale import *
> ?File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
> ? from ctypes import (Structure, POINTER, create_string_buffer,
> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
> ? _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
> ?File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
> ? ffiargs, ffires, self._flags_)
> MemoryError
>
>
> OS: Mac OS Snow Leopard 10.6.5
>
> I tried to sign up your bugtracker to report a bug but the registration is broken ?-- I got ?the error message when I was trying to confirm my registration.
> My login is prykhodchenko
>
>
> ---
> Roman Prykhodchenko
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From fijall at gmail.com  Sat Nov 20 09:10:21 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 20 Nov 2010 10:10:21 +0200
Subject: [pypy-dev] Needs some guidance in writing the kqueue part of
	select
In-Reply-To: <AANLkTinLJx3w1ksF2PorkJA+r=fMq92hMKEaMQmogkDq@mail.gmail.com>
References: <AANLkTinLJx3w1ksF2PorkJA+r=fMq92hMKEaMQmogkDq@mail.gmail.com>
Message-ID: <AANLkTi=3R_YhEHvon_T1z6DLW6oHeKFNqXyz6G6D+3th@mail.gmail.com>

(part response)

>
> 2) How do I declare that an argument is const in rffi?
>

usually you don't


From lac at openend.se  Sat Nov 20 16:17:11 2010
From: lac at openend.se (Laura Creighton)
Date: Sat, 20 Nov 2010 16:17:11 +0100
Subject: [pypy-dev] Conference in Japan
Message-ID: <201011201517.oAKFHBJI022329@theraft.openend.se>


------- Forwarded Message
From: <Andreas.Schreiber at dlr.de>
To: <python-announce-list at python.org>
Subject: [ANN] Call for Papers: Workshop Python for High Performance and  Scientific Computing
Thread-Topic: [ANN] Call for Papers: Workshop Python for High Performance
	and Scientific Computing
Thread-Index: AcuIOaLheS34scrKT/anKkKcPOGEAA==
Date: Fri, 19 Nov 2010 22:32:23 +0000
Message-ID: <64886ABF09B1924997D70320E02F7DDB03F1AE at DLREXMBX01.intra.dlr.de>

Workshop Python for High Performance and Scientific Computing


June 1-3, 2011, Tsukuba, Japan

part of The International Conference on Computational Science (ICCS 2011)

http://www.dlr.de/sc/iccs2011

===============
Call for Papers
===============

Introduction
- ------------
Python is an accepted high-level programming language with a growing
community in academia and industry. Beside its original use as a
scripting language for web applications, today Python is a
general-purpose language adopted by many scientific applications like
CFD, bio molecular simulation, AI, scientific visualization etc. More
and more industrial domains are turning towards it as well, such as
robotics, semiconductor manufacturing, automotive solutions,
telecommunication, computer graphics, and games. In all fields, the
use of Python for scientific, high performance parallel, and
distributed computing, as well as general scripted automation is
increasing. Moreover, Python is well-suited for education in
scientific computing.

The workshop aims at bringing together researchers and practitioners
from industry and academia using Python for all aspects of high
performance and scientific computing. The goal is to present
Python-based scientific applications and libraries, to discuss general
topics regarding the use of Python (such as language design and
performance issues), and to share experience using Python in
scientific computing education.

Topics
- ------
    * Python-based scientific applications and libraries
    * High performance computing
    * Parallel Python-based programming languages
    * Scientific visualization
    * Scientific computing education
    * Python performance and language issues
    * Problem solving environments with Python
    * Performance analysis tools for Python application

Papers/Submission
- -----------------
We invite you to submit a paper of up to 10 pages formatted according
to the rules of Procedia Computer Science
(http://www.elsevier.com/locate/inca/719435) via the ICCS 2011
conference submission system
(http://www.iccs-meeting.org/iccs2011/papers/upload.php) selecting our
workshop Python for High Performance and Scientific Computing in the
drop-down menu.

Important Dates
- ---------------
Full paper submission:        January 10, 2011
Notification of acceptance:   February 20, 2011
Camera-ready papers:          March 7, 2011


Program Committee
- -----------------
    * Achim Basermann, German Aerospace Center, Germany
    * David Beazley, Dabeaz, LLC, USA
    * William E. Hart, Sandia National Laboratories, USA
    * Konrad Hinsen, Centre de Biophysique Mol?culaire, CNRS Orl?ans, France
    * Andreas Kl?ckner, New York University, USA
    * Maurice Ling, Singapore Polytechnic, Singapore
    * Stuart Mitchell, The University of Auckland, New Zealand
    * Mike M?ller, Python Academy, Germany
    * Travis Oliphant, Enthought, Inc., USA
    * Fernando P?rez, University of California, Berkeley, USA
    * Massimo Di Pierro, DePaul University, USA
    * Marc Poinot, ONERA, France
    * William Scullin, Argonne National Laboratory, USA 
    * Ga?l Varoquaux, INRIA, France

Workshop Organizers
- -------------------
    * Chair:   Andreas Schreiber, German Aerospace Center (DLR), Germany
               E-Mail: iccs2011 at dlr.de
    * Co-Chair:  Guy K. Kloss, Auckland University of Technology, New Zealand




- --
Andreas Schreiber
Head Distributed Systems and Component Software
DLR (German Aerospace Center), Simulation and Software Technology
Linder Hoehe, 51147 Cologne, Germany
Tel: +49 (0)2203/601-2485  http://www.dlr.de/sc
Fax: +49 (0)2203/601-12485  MSN: mailto:Andreas.Schreiber at dlr.de
Mobile: +49 (0)173 5231013 ICQ# 324185855

DLR on Twitter: http://twitter.com/DLR_de
- -- 
http://mail.python.org/mailman/listinfo/python-announce-list

        Support the Python Software Foundation:
        http://www.python.org/psf/donations/

------- End of Forwarded Message



From romcheg.prihod at gmail.com  Sat Nov 20 20:20:32 2010
From: romcheg.prihod at gmail.com (Roman Prykhodchenko)
Date: Sat, 20 Nov 2010 21:20:32 +0200
Subject: [pypy-dev] Memory error when importing optparse module
In-Reply-To: <AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>
References: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
	<AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>
Message-ID: <DECFEB32-E803-4BB0-BF43-7D37CC4B70DE@gmail.com>

I built the executable using macports (port install pypy).
My python is 64 bit.

I can help you to find the bug if you help me to help you.


- Roman

On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote:

> Hey.
> 
> Unfortunately we don't have a full-time OS X developer and it makes it
> harder for this platform to be fully spported. Is your python 32bit or
> 64bit? How did you build that executable?
> 
> On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko
> <romcheg.prihod at gmail.com> wrote:
>> Hi guys,
>> 
>> 
>> I'm trying to port my scripts to pypy but when I import the optparse module I get the error:
>> 
>>>>>> import optparse
>> Traceback (most recent call last):
>>  File "<console>", line 1, in <module>
>>  File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
>>   from gettext import gettext
>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
>>   import locale, copy, os, re, struct, sys
>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
>>   from _locale import *
>>  File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
>>   from ctypes import (Structure, POINTER, create_string_buffer,
>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
>>   _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
>>  File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
>>   ffiargs, ffires, self._flags_)
>> MemoryError
>> 
>> 
>> OS: Mac OS Snow Leopard 10.6.5
>> 
>> I tried to sign up your bugtracker to report a bug but the registration is broken  -- I got  the error message when I was trying to confirm my registration.
>> My login is prykhodchenko
>> 
>> 
>> ---
>> Roman Prykhodchenko
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>> 



From santagada at gmail.com  Sun Nov 21 00:54:32 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 20 Nov 2010 21:54:32 -0200
Subject: [pypy-dev] Needs some guidance in writing the kqueue part of
	select
In-Reply-To: <AANLkTi=3R_YhEHvon_T1z6DLW6oHeKFNqXyz6G6D+3th@mail.gmail.com>
References: <AANLkTinLJx3w1ksF2PorkJA+r=fMq92hMKEaMQmogkDq@mail.gmail.com>
	<AANLkTi=3R_YhEHvon_T1z6DLW6oHeKFNqXyz6G6D+3th@mail.gmail.com>
Message-ID: <AANLkTim71Faqvw5dK1erMQO0EdR4ss5nf=0erTcBjA35@mail.gmail.com>

here is the latest version of the code http://paste.pocoo.org/show/294079/

Things apear to be working except that when I run the tests
http://paste.pocoo.org/show/294087/ the kevent one fails. Supposedly
it should give me back the event setting kevent.data to how much you
can read on the file and it is returning 0 to me.

Maybe I did something dumb while declaring kevent_array or in
kevent_struct, but I'm kind of lost. Python 3.1 show the expected
behavior (of course, using its much more high level interface).

Any tips?


-- 
Leonardo Santagada


From p.giarrusso at gmail.com  Sun Nov 21 10:26:04 2010
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sun, 21 Nov 2010 10:26:04 +0100
Subject: [pypy-dev] Needs some guidance in writing the kqueue part of
	select
In-Reply-To: <AANLkTim71Faqvw5dK1erMQO0EdR4ss5nf=0erTcBjA35@mail.gmail.com>
References: <AANLkTinLJx3w1ksF2PorkJA+r=fMq92hMKEaMQmogkDq@mail.gmail.com>
	<AANLkTi=3R_YhEHvon_T1z6DLW6oHeKFNqXyz6G6D+3th@mail.gmail.com>
	<AANLkTim71Faqvw5dK1erMQO0EdR4ss5nf=0erTcBjA35@mail.gmail.com>
Message-ID: <AANLkTim=xBh4mTW_TA97EgQ2+0tdZsD9q3M37Ty34aby@mail.gmail.com>

On Sun, Nov 21, 2010 at 00:54, Leonardo Santagada <santagada at gmail.com> wrote:
> here is the latest version of the code http://paste.pocoo.org/show/294079/
>
> Things apear to be working except that when I run the tests
> http://paste.pocoo.org/show/294087/ the kevent one fails. Supposedly
> it should give me back the event setting kevent.data to how much you
> can read on the file and it is returning 0 to me.
>
> Maybe I did something dumb while declaring kevent_array or in
> kevent_struct, but I'm kind of lost. Python 3.1 show the expected
> behavior (of course, using its much more high level interface).

> Any tips?

Have you tried using strace on the two programs, to see and compare
the actual invocations?
Variations of strace -e kqueue,kevent,file,network might help
selecting relevant data (-e syscall1,syscall2,...,syscallN shows only
those syscalls; file and network are syscall categories).

If you are not on Linux but on other *BSDes (and it seems you must be,
since Linux has other syscalls), you might still use strace or
equivalents:
http://www.cyberciti.biz/tips/ktrace-freebsd-macosx-tool-howto.html

Best regards
-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/


From fijall at gmail.com  Sun Nov 21 13:20:59 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 21 Nov 2010 14:20:59 +0200
Subject: [pypy-dev] Memory error when importing optparse module
In-Reply-To: <DECFEB32-E803-4BB0-BF43-7D37CC4B70DE@gmail.com>
References: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
	<AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>
	<DECFEB32-E803-4BB0-BF43-7D37CC4B70DE@gmail.com>
Message-ID: <AANLkTikpQh33ZXsmLRsA2kJoHGwDQFN6qfBLmzd5Rffj@mail.gmail.com>

I guess the first thing would be to build pypy svn yourself instead of
using 1.3 and see if it helps

On Sat, Nov 20, 2010 at 9:20 PM, Roman Prykhodchenko
<romcheg.prihod at gmail.com> wrote:
> I built the executable using macports (port install pypy).
> My python is 64 bit.
>
> I can help you to find the bug if you help me to help you.
>
>
> - Roman
>
> On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote:
>
>> Hey.
>>
>> Unfortunately we don't have a full-time OS X developer and it makes it
>> harder for this platform to be fully spported. Is your python 32bit or
>> 64bit? How did you build that executable?
>>
>> On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko
>> <romcheg.prihod at gmail.com> wrote:
>>> Hi guys,
>>>
>>>
>>> I'm trying to port my scripts to pypy but when I import the optparse module I get the error:
>>>
>>>>>>> import optparse
>>> Traceback (most recent call last):
>>> ?File "<console>", line 1, in <module>
>>> ?File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
>>> ? from gettext import gettext
>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
>>> ? import locale, copy, os, re, struct, sys
>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
>>> ? from _locale import *
>>> ?File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
>>> ? from ctypes import (Structure, POINTER, create_string_buffer,
>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
>>> ? _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
>>> ?File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
>>> ? ffiargs, ffires, self._flags_)
>>> MemoryError
>>>
>>>
>>> OS: Mac OS Snow Leopard 10.6.5
>>>
>>> I tried to sign up your bugtracker to report a bug but the registration is broken ?-- I got ?the error message when I was trying to confirm my registration.
>>> My login is prykhodchenko
>>>
>>>
>>> ---
>>> Roman Prykhodchenko
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>
>


From romcheg.prihod at gmail.com  Sun Nov 21 13:31:59 2010
From: romcheg.prihod at gmail.com (Roman Prykhodchenko)
Date: Sun, 21 Nov 2010 14:31:59 +0200
Subject: [pypy-dev] Memory error when importing optparse module
In-Reply-To: <AANLkTikpQh33ZXsmLRsA2kJoHGwDQFN6qfBLmzd5Rffj@mail.gmail.com>
References: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
	<AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>
	<DECFEB32-E803-4BB0-BF43-7D37CC4B70DE@gmail.com>
	<AANLkTikpQh33ZXsmLRsA2kJoHGwDQFN6qfBLmzd5Rffj@mail.gmail.com>
Message-ID: <25355127-9CE4-4DD8-B902-DCDD3B9CFB82@gmail.com>

I have already built it myself. I got lots of errors and warnings during the build process and as a result I have pypy-c that doesn't work.
May be that was caused by --opt=jit on my x64 platform.
Will try to go with --opt=2 a bit later and report you the result.


- Roman




On Nov 21, 2010, at 2:20 PM, Maciej Fijalkowski wrote:

> I guess the first thing would be to build pypy svn yourself instead of
> using 1.3 and see if it helps
> 
> On Sat, Nov 20, 2010 at 9:20 PM, Roman Prykhodchenko
> <romcheg.prihod at gmail.com> wrote:
>> I built the executable using macports (port install pypy).
>> My python is 64 bit.
>> 
>> I can help you to find the bug if you help me to help you.
>> 
>> 
>> - Roman
>> 
>> On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote:
>> 
>>> Hey.
>>> 
>>> Unfortunately we don't have a full-time OS X developer and it makes it
>>> harder for this platform to be fully spported. Is your python 32bit or
>>> 64bit? How did you build that executable?
>>> 
>>> On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko
>>> <romcheg.prihod at gmail.com> wrote:
>>>> Hi guys,
>>>> 
>>>> 
>>>> I'm trying to port my scripts to pypy but when I import the optparse module I get the error:
>>>> 
>>>>>>>> import optparse
>>>> Traceback (most recent call last):
>>>>  File "<console>", line 1, in <module>
>>>>  File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
>>>>   from gettext import gettext
>>>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
>>>>   import locale, copy, os, re, struct, sys
>>>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
>>>>   from _locale import *
>>>>  File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
>>>>   from ctypes import (Structure, POINTER, create_string_buffer,
>>>>  File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
>>>>   _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
>>>>  File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
>>>>   ffiargs, ffires, self._flags_)
>>>> MemoryError
>>>> 
>>>> 
>>>> OS: Mac OS Snow Leopard 10.6.5
>>>> 
>>>> I tried to sign up your bugtracker to report a bug but the registration is broken  -- I got  the error message when I was trying to confirm my registration.
>>>> My login is prykhodchenko
>>>> 
>>>> 
>>>> ---
>>>> Roman Prykhodchenko
>>>> _______________________________________________
>>>> pypy-dev at codespeak.net
>>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>> 
>> 
>> 



From fijall at gmail.com  Sun Nov 21 13:38:33 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 21 Nov 2010 14:38:33 +0200
Subject: [pypy-dev] Memory error when importing optparse module
In-Reply-To: <25355127-9CE4-4DD8-B902-DCDD3B9CFB82@gmail.com>
References: <A679AA31-AD7F-4CD3-8AAF-97E077EB3B34@gmail.com>
	<AANLkTiny7U1qbTQMGNOKqLfe33TajtfV8Zuskrt19+tb@mail.gmail.com>
	<DECFEB32-E803-4BB0-BF43-7D37CC4B70DE@gmail.com>
	<AANLkTikpQh33ZXsmLRsA2kJoHGwDQFN6qfBLmzd5Rffj@mail.gmail.com>
	<25355127-9CE4-4DD8-B902-DCDD3B9CFB82@gmail.com>
Message-ID: <AANLkTimMFxDgkmgetpvg5+yaPuYGRBCNgkMtts=dVZCO@mail.gmail.com>

PyPy is known not to work on mac os x 64bit. Blame a combination of
crazy licensing schemes (we can't run a buildbot in a virtual machine
nor we can debug it) with awkward outdated tools available on
platform. Or blame us :) Generally someone has to fix this, but as I
said before we don't have an OS X developer. Precise problem is some
difference in generated assembler by gcc or platform needs.

On Sun, Nov 21, 2010 at 2:31 PM, Roman Prykhodchenko
<romcheg.prihod at gmail.com> wrote:
> I have already built it myself. I got lots of errors and warnings during the build process and as a result I have pypy-c that doesn't work.
> May be that was caused by --opt=jit on my x64 platform.
> Will try to go with --opt=2 a bit later and report you the result.
>
>
> - Roman
>
>
>
>
> On Nov 21, 2010, at 2:20 PM, Maciej Fijalkowski wrote:
>
>> I guess the first thing would be to build pypy svn yourself instead of
>> using 1.3 and see if it helps
>>
>> On Sat, Nov 20, 2010 at 9:20 PM, Roman Prykhodchenko
>> <romcheg.prihod at gmail.com> wrote:
>>> I built the executable using macports (port install pypy).
>>> My python is 64 bit.
>>>
>>> I can help you to find the bug if you help me to help you.
>>>
>>>
>>> - Roman
>>>
>>> On Nov 20, 2010, at 10:09 AM, Maciej Fijalkowski wrote:
>>>
>>>> Hey.
>>>>
>>>> Unfortunately we don't have a full-time OS X developer and it makes it
>>>> harder for this platform to be fully spported. Is your python 32bit or
>>>> 64bit? How did you build that executable?
>>>>
>>>> On Fri, Nov 19, 2010 at 9:46 PM, Roman Prykhodchenko
>>>> <romcheg.prihod at gmail.com> wrote:
>>>>> Hi guys,
>>>>>
>>>>>
>>>>> I'm trying to port my scripts to pypy but when I import the optparse module I get the error:
>>>>>
>>>>>>>>> import optparse
>>>>> Traceback (most recent call last):
>>>>> ?File "<console>", line 1, in <module>
>>>>> ?File "/opt/local/share/pypy-1.3/lib-python/2.5.2/optparse.py", line 84, in <module>
>>>>> ? from gettext import gettext
>>>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/gettext.py", line 49, in <module>
>>>>> ? import locale, copy, os, re, struct, sys
>>>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/locale.py", line 30, in <module>
>>>>> ? from _locale import *
>>>>> ?File "/opt/local/share/pypy-1.3/pypy/lib/_locale.py", line 6, in <module>
>>>>> ? from ctypes import (Structure, POINTER, create_string_buffer,
>>>>> ?File "/opt/local/share/pypy-1.3/lib-python/modified-2.5.2/ctypes/__init__.py", line 489, in <module>
>>>>> ? _cast = PYFUNCTYPE(py_object, c_void_p, py_object, py_object)(_cast_addr)
>>>>> ?File "/opt/local/share/pypy-1.3/pypy/lib/_ctypes/function.py", line 104, in __init__
>>>>> ? ffiargs, ffires, self._flags_)
>>>>> MemoryError
>>>>>
>>>>>
>>>>> OS: Mac OS Snow Leopard 10.6.5
>>>>>
>>>>> I tried to sign up your bugtracker to report a bug but the registration is broken ?-- I got ?the error message when I was trying to confirm my registration.
>>>>> My login is prykhodchenko
>>>>>
>>>>>
>>>>> ---
>>>>> Roman Prykhodchenko
>>>>> _______________________________________________
>>>>> pypy-dev at codespeak.net
>>>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>>>
>>>
>>>
>
>


From anto.cuni at gmail.com  Tue Nov 23 08:31:36 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 23 Nov 2010 08:31:36 +0100
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <20101123071858.17721282BD6@codespeak.net>
References: <20101123071858.17721282BD6@codespeak.net>
Message-ID: <4CEB6DD8.5080504@gmail.com>

On 23/11/10 08:18, fijal at codespeak.net wrote:
> Author: fijal
> Date: Tue Nov 23 08:18:57 2010
> New Revision: 79382
>
> Modified:
>     pypy/trunk/pypy/config/translationoption.py
> Log:
> Disable jit debugging by default
>
>
> Modified: pypy/trunk/pypy/config/translationoption.py
> ==============================================================================
> --- pypy/trunk/pypy/config/translationoption.py	(original)
> +++ pypy/trunk/pypy/config/translationoption.py	Tue Nov 23 08:18:57 2010
> @@ -115,7 +115,7 @@
>                    default="auto", cmdline="--jit-backend"),
>       ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT",
>                    ["off", "profile", "steps", "detailed"],
> -                 default="profile",      # XXX for now
> +                 default="off",
>                    cmdline="--jit-debug"),
>       ChoiceOption("jit_profiler", "integrate profiler support into the JIT",
>                    ["off", "oprofile"],

Not sure it's a good idea. It's useful to see the statistics at the end of the 
run.  I know that you can turn it on explicitly, but it's easy to forget.

What about having an env variable that automatically turns jit debug on?

ciao,
Anto


From arigo at tunes.org  Tue Nov 23 17:52:18 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Nov 2010 17:52:18 +0100
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <4CEB6DD8.5080504@gmail.com>
References: <20101123071858.17721282BD6@codespeak.net>
	<4CEB6DD8.5080504@gmail.com>
Message-ID: <AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>

Hi Anto,

On Tue, Nov 23, 2010 at 8:31 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> Disable jit debugging by default
>
> Not sure it's a good idea. It's useful to see the statistics at the end of the
> run. ?I know that you can turn it on explicitly, but it's easy to forget.
>
> What about having an env variable that automatically turns jit debug on?

I guess it should just be another log category, like "jit-summary".
Then you can get the effect you want by setting (in the shell) PYPYLOG
to "jit-summary:-".


A bient?t,

Armin.


From fijall at gmail.com  Tue Nov 23 21:31:11 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 23 Nov 2010 22:31:11 +0200
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>
References: <20101123071858.17721282BD6@codespeak.net>
	<4CEB6DD8.5080504@gmail.com>
	<AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>
Message-ID: <AANLkTimPFJGSCCKo-HyCkCVqTXrWUsJnwgRR6FAg3JrL@mail.gmail.com>

On Tue, Nov 23, 2010 at 6:52 PM, Armin Rigo <arigo at tunes.org> wrote:> Hi Anto,
>
> On Tue, Nov 23, 2010 at 8:31 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>>> Disable jit debugging by default
>>
>> Not sure it's a good idea. It's useful to see the statistics at the end of the
>> run. ?I know that you can turn it on explicitly, but it's easy to forget.
>>
>> What about having an env variable that automatically turns jit debug on?
>
> I guess it should just be another log category, like "jit-summary".
> Then you can get the effect you want by setting (in the shell) PYPYLOG
> to "jit-summary:-".
>

This or an env variable. I find it really disruptive for everyday
usage. Like py.test -k test_<tab> displaying jit log is not exactly
what I want. You can turn it on, put an alias or log it somewhere.


From wlavrijsen at lbl.gov  Tue Nov 23 21:48:05 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 23 Nov 2010 12:48:05 -0800 (PST)
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
Message-ID: <alpine.LNX.2.00.1011231245320.12126@pc3-wlav.lbl.gov>

Hi,

I've been banging my head on the error below. I've removed the references
to lltype.direct_ptradd in my code that I'm aware of, but that didn't do
the trick. Any ideas, or perhaps an implementation of bhimpl_direct_ptradd?

Thanks,
      Wim

[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "../../../translator/goal/translate.py", line 290, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/translator/driver.py", line 817, in proceed
[translation:ERROR]     return self._execute(goals, task_skip = self._maybe_skip())
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/translator/tool/taskengine.py", line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/translator/driver.py", line 293, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/translator/driver.py", line 403, in task_pyjitpl_lltype
[translation:ERROR]     backend_name=self.config.translation.jit_backend, inline=True)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/warmspot.py", line 47, in apply_jit
[translation:ERROR]     **kwds)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/warmspot.py", line 188, in __init__
[translation:ERROR]     self.metainterp_sd.finish_setup(self.codewriter, optimizer=optimizer)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/pyjitpl.py", line 1268, in finish_setup
[translation:ERROR]     self.blackholeinterpbuilder = BlackholeInterpBuilder(codewriter, self)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/blackhole.py", line 45, in __init__
[translation:ERROR]     self.setup_insns(asm.insns)
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/blackhole.py", line 67, in setup_insns
[translation:ERROR]     all_funcs.append(self._get_method(name, argcodes))
[translation:ERROR]    File "/home/wlav/pydev/pypy-reflex/pypy/jit/metainterp/blackhole.py", line 223, in _get_method
[translation:ERROR]     unboundmethod = getattr(BlackholeInterpreter, 'bhimpl_' + name).im_func
[translation:ERROR]  AttributeError: type object 'BlackholeInterpreter' has no attribute 'bhimpl_direct_ptradd'

--
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From drsalists at gmail.com  Wed Nov 24 01:13:16 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 23 Nov 2010 16:13:16 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CE54103.6060603@gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
	<4CE54103.6060603@gmail.com>
Message-ID: <AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>

On Thu, Nov 18, 2010 at 7:06 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> On 18/11/10 01:17, Dan Stromberg wrote:
>
> Hi Dan,
>
> [cut]
>
>  Given #1 and #2 above, anydbm should continue working, due to the presence
>> of
>> gdbm and dumbdbm.
>>
>> I guess I think that if someone has a need for bsddb (and it's assorted
>> interfaces), they probably should work on that.
>>
>> Sound reasonable?
>>
>
> Yes. To me it sounds reasonable. I don't know if it's correct because I
> never used any of those modules, but it's surely reasoable :-).
>
> I think that the best would be if you get an account on codespeak and do
> that in a branch: this way we can follow your commits and the if everything
> is fine we merge it.
>
> I personally cannot create new accounts on codespeak, so you should ask
> someone else for this (e.g. Armin Rigo): the easiest way is if you just come
> on #pypy on freenode.net and talk with us "real-time".
>
> ciao,
> Anto
>

I've created a branch at http://codespeak.net/svn/pypy/branch/gdbm and made
the proposed changes in it.

I took the liberty of creating an "attic" directory to put dbm.py in.

Please let me know if there are further changes required related to this.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101123/9a986426/attachment.html>

From exarkun at twistedmatrix.com  Wed Nov 24 01:19:42 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 24 Nov 2010 00:19:42 -0000
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <AANLkTimPFJGSCCKo-HyCkCVqTXrWUsJnwgRR6FAg3JrL@mail.gmail.com>
References: <20101123071858.17721282BD6@codespeak.net>
	<4CEB6DD8.5080504@gmail.com>
	<AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>
	<AANLkTimPFJGSCCKo-HyCkCVqTXrWUsJnwgRR6FAg3JrL@mail.gmail.com>
Message-ID: <20101124001942.2109.1155721242.divmod.xquotient.188@localhost.localdomain>

On 23 Nov, 08:31 pm, fijall at gmail.com wrote:
>On Tue, Nov 23, 2010 at 6:52 PM, Armin Rigo <arigo at tunes.org> wrote:> 
>Hi Anto,
>>
>>On Tue, Nov 23, 2010 at 8:31 AM, Antonio Cuni <anto.cuni at gmail.com> 
>>wrote:
>>>>Disable jit debugging by default
>>>
>>>Not sure it's a good idea. It's useful to see the statistics at the 
>>>end of the
>>>run. ?I know that you can turn it on explicitly, but it's easy to 
>>>forget.
>>>
>>>What about having an env variable that automatically turns jit debug 
>>>on?
>>
>>I guess it should just be another log category, like "jit-summary".
>>Then you can get the effect you want by setting (in the shell) PYPYLOG
>>to "jit-summary:-".
>
>This or an env variable. I find it really disruptive for everyday
>usage. Like py.test -k test_<tab> displaying jit log is not exactly
>what I want. You can turn it on, put an alias or log it somewhere.

It also breaks Twisted unit tests, which spawn sys.executable as a child 
and then inspect output in various ways.  It would be nice to be able to 
support the jit-summary-by-default behavior, but that probably involves 
adding something like a `sys.executablev` list that contains the 
executable and extra arguments to pass to the executable, so that the 
summary can *really* be turned off.

Jean-Paul


From cfbolz at gmx.de  Wed Nov 24 09:41:12 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 24 Nov 2010 09:41:12 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>	<4CE2765E.2020507@gmail.com>	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>	<4CE54103.6060603@gmail.com>
	<AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>
Message-ID: <4CECCFA8.80708@gmx.de>

On 11/24/2010 01:13 AM, Dan Stromberg wrote:
> I've created a branch at http://codespeak.net/svn/pypy/branch/gdbm and
> made the proposed changes in it.
>
> I took the liberty of creating an "attic" directory to put dbm.py in.

Any reason not to just delete dbm.py? This attic stuff is annoying, and 
if necessary you can still get the file from svn history.

Cheers,

Carl Friedrich


From cfbolz at gmx.de  Wed Nov 24 10:33:48 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 24 Nov 2010 10:33:48 +0100
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
In-Reply-To: <alpine.LNX.2.00.1011231245320.12126@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011231245320.12126@pc3-wlav.lbl.gov>
Message-ID: <4CECDBFC.60706@gmx.de>

On 11/23/2010 09:48 PM, wlavrijsen at lbl.gov wrote:
> I've been banging my head on the error below. I've removed the references
> to lltype.direct_ptradd in my code that I'm aware of, but that didn't do
> the trick.

I told the JIT not to look into TypeConverter._get_fieldptr, see 
revision 79444. Afterwards it seemed to work.

> Any ideas, or perhaps an implementation of bhimpl_direct_ptradd?

Ideally support should be added to the JIT, yes (Armin? :-) ). But for 
now the above solution should work too.

Cheers,

Carl Friedrich


From arigo at tunes.org  Wed Nov 24 13:39:18 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 24 Nov 2010 13:39:18 +0100
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <20101124001942.2109.1155721242.divmod.xquotient.188@localhost.localdomain>
References: <20101123071858.17721282BD6@codespeak.net>
	<4CEB6DD8.5080504@gmail.com>
	<AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>
	<AANLkTimPFJGSCCKo-HyCkCVqTXrWUsJnwgRR6FAg3JrL@mail.gmail.com>
	<20101124001942.2109.1155721242.divmod.xquotient.188@localhost.localdomain>
Message-ID: <AANLkTikCKNAP4cPjZeMSKaR0u2KbKOWo_LjfjUcPy-UN@mail.gmail.com>

Hi,

On Wed, Nov 24, 2010 at 1:19 AM,  <exarkun at twistedmatrix.com> wrote:
>>> I guess it should just be another log category, like "jit-summary".
>>> Then you can get the effect you want by setting (in the shell) PYPYLOG
>>> to "jit-summary:-".
>>
>> This or an env variable.

The PYPYLOG solution I described *is* an environment variable.  What
do you mean?


A bient?t,

Armin.


From fijall at gmail.com  Wed Nov 24 13:41:47 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 24 Nov 2010 14:41:47 +0200
Subject: [pypy-dev] [pypy-svn] r79382 - pypy/trunk/pypy/config
In-Reply-To: <AANLkTikCKNAP4cPjZeMSKaR0u2KbKOWo_LjfjUcPy-UN@mail.gmail.com>
References: <20101123071858.17721282BD6@codespeak.net>
	<4CEB6DD8.5080504@gmail.com>
	<AANLkTinw5QWFrtO73Tw9ayJND99_ASmEsis7+fMowR9w@mail.gmail.com>
	<AANLkTimPFJGSCCKo-HyCkCVqTXrWUsJnwgRR6FAg3JrL@mail.gmail.com>
	<20101124001942.2109.1155721242.divmod.xquotient.188@localhost.localdomain>
	<AANLkTikCKNAP4cPjZeMSKaR0u2KbKOWo_LjfjUcPy-UN@mail.gmail.com>
Message-ID: <AANLkTimR6OuUct1EuTUwPeRDqn6ShPqehQabJm57+JWa@mail.gmail.com>

On Wed, Nov 24, 2010 at 2:39 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi,
>
> On Wed, Nov 24, 2010 at 1:19 AM, ?<exarkun at twistedmatrix.com> wrote:
>>>> I guess it should just be another log category, like "jit-summary".
>>>> Then you can get the effect you want by setting (in the shell) PYPYLOG
>>>> to "jit-summary:-".
>>>
>>> This or an env variable.
>
> The PYPYLOG solution I described *is* an environment variable. ?What
> do you mean?
>

yes of course. nothing any more :)


From drsalists at gmail.com  Wed Nov 24 18:04:18 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 24 Nov 2010 09:04:18 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CECCFA8.80708@gmx.de>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
	<4CE54103.6060603@gmail.com>
	<AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>
	<4CECCFA8.80708@gmx.de>
Message-ID: <AANLkTi=tZ4ELPY19UaNkVPHPshSea0h5ERH5QTeRNEYE@mail.gmail.com>

One tiny reason: People tend to forget that they can pull things from SVN
history, when an isolated file disappears.  I'm expecting someday someone's
going to want to add bsddb - when they do, dbm.py might be a decent starting
point (but it likely has memory leaks that were corrected in gdbm.py).

I'd be happy to delete it if that's the consensus.

On Wed, Nov 24, 2010 at 12:41 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:

> On 11/24/2010 01:13 AM, Dan Stromberg wrote:
> > I've created a branch at http://codespeak.net/svn/pypy/branch/gdbm and
> > made the proposed changes in it.
> >
> > I took the liberty of creating an "attic" directory to put dbm.py in.
>
> Any reason not to just delete dbm.py? This attic stuff is annoying, and
> if necessary you can still get the file from svn history.
>
> Cheers,
>
> Carl Friedrich
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/15aa92b3/attachment.html>

From exarkun at twistedmatrix.com  Wed Nov 24 19:50:15 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Wed, 24 Nov 2010 18:50:15 -0000
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=tZ4ELPY19UaNkVPHPshSea0h5ERH5QTeRNEYE@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
	<4CE54103.6060603@gmail.com>
	<AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>
	<4CECCFA8.80708@gmx.de>
	<AANLkTi=tZ4ELPY19UaNkVPHPshSea0h5ERH5QTeRNEYE@mail.gmail.com>
Message-ID: <20101124185015.2109.1131897446.divmod.xquotient.218@localhost.localdomain>

On 05:04 pm, drsalists at gmail.com wrote:
>One tiny reason: People tend to forget that they can pull things from 
>SVN
>history, when an isolated file disappears.  I'm expecting someday 
>someone's
>going to want to add bsddb - when they do, dbm.py might be a decent 
>starting
>point (but it likely has memory leaks that were corrected in gdbm.py).

Anyone who cannot figure out how to use svn has a negligible chance of 
successfully implementing the bsddb module.

Jean-Paul


From drsalists at gmail.com  Wed Nov 24 19:56:42 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 24 Nov 2010 10:56:42 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <20101124185015.2109.1131897446.divmod.xquotient.218@localhost.localdomain>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<AANLkTikzsFEYwqD9RB6KyDfh67Tijr88kKfZFwBvWp47@mail.gmail.com>
	<4CE2765E.2020507@gmail.com>
	<AANLkTi=C-bByo0dPoD11=aCw-YVFkkXqKLU_71SjyEzS@mail.gmail.com>
	<4CE54103.6060603@gmail.com>
	<AANLkTi=vGPOm=HQtqc182c1Gv9_9019NYwj_hLnSjdDx@mail.gmail.com>
	<4CECCFA8.80708@gmx.de>
	<AANLkTi=tZ4ELPY19UaNkVPHPshSea0h5ERH5QTeRNEYE@mail.gmail.com>
	<20101124185015.2109.1131897446.divmod.xquotient.218@localhost.localdomain>
Message-ID: <AANLkTin9V2QJBvHwYq_UkxgfpDnh+5c51cZ7FuD7-J6Z@mail.gmail.com>

On Wed, Nov 24, 2010 at 10:50 AM, <exarkun at twistedmatrix.com> wrote:

> On 05:04 pm, drsalists at gmail.com wrote:
> >One tiny reason: People tend to forget that they can pull things from
> >SVN
> >history, when an isolated file disappears.  I'm expecting someday
> >someone's
> >going to want to add bsddb - when they do, dbm.py might be a decent
> >starting
> >point (but it likely has memory leaks that were corrected in gdbm.py).
>
> Anyone who cannot figure out how to use svn has a negligible chance of
> successfully implementing the bsddb module.
>

It's not really a matter of not knowing how to use SVN.  Peg revisions are
easy enough - even people who don't know them yet, will probably learn them
easily.  But if no one points out that there was once something vaguely
related to what they want to do in the present,...  Well, I think few would
know to look for it under the name of "dbm.py", even if it occurred to them
to look backward in time before getting started.  That is, unless they saw
this thread in the archives :)

But I'm starting to get the feeling that the consensus is that it should be
deleted.  That's not a problem.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/05cb0a2a/attachment.html>

From wlavrijsen at lbl.gov  Wed Nov 24 23:41:40 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 24 Nov 2010 14:41:40 -0800 (PST)
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
In-Reply-To: <4CECDBFC.60706@gmx.de>
References: <alpine.LNX.2.00.1011231245320.12126@pc3-wlav.lbl.gov>
	<4CECDBFC.60706@gmx.de>
Message-ID: <alpine.LNX.2.00.1011241414120.12126@pc3-wlav.lbl.gov>

Hi Carl,

> I told the JIT not to look into TypeConverter._get_fieldptr, see
> revision 79444. Afterwards it seemed to work.

yes ... I now finally have my executable again. :)

Some initial numbers from bench1, which we ran during the sprint, are (and
they are not as accurate as the digits seem to presume):

PyROOT:   62.8
PyCintex: 84.9
pypy-c:   11.0
C++:       0.05

For reminder: PyCintex is Reflex + PyROOT; the number is the difference between
an empty loop and one calling into a C++ function that basically does nothing.
I'm a little surprised b/c I remember that PyROOT was only 2x slower when we
measured during the sprint, but that might simply be improvement in the JIT of
course, and I don't remember the C++ data point to anchor it (although I thought
it was lower than a factor 200), so oh well.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From wlavrijsen at lbl.gov  Thu Nov 25 01:32:20 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 24 Nov 2010 16:32:20 -0800 (PST)
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
Message-ID: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>

Hi,

so I'm revising my numbers after finding out that I was using a debug version
of ROOT/Reflex ...

PyROOT:   48.6
PyCintex: 50.2
pypy-c:    5.5
C++:       0.05

Note that although PyROOT speeds up only marginally in opt mode, pypy-c has
a huge speedup. For PyROOT, this is b/c most of the time is spent in CPython
through API calls and that was already in opt mode. For pypy-c, this is
telling me quite clearly that I've got to work some on capi.py. ;)

Of course, the factor 6 that I saw originally (v.s. 2 during the sprint) is
now a factor 10 almost ...

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From fijall at gmail.com  Thu Nov 25 07:40:16 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Nov 2010 08:40:16 +0200
Subject: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
In-Reply-To: <20101124163904.C308C50810@codespeak.net>
References: <20101124163904.C308C50810@codespeak.net>
Message-ID: <AANLkTikvrsprwj9ehh8ZRndDvqKCYY7zms4um6uEqHCc@mail.gmail.com>

Is pypy repository really a place for such files? Maybe we should keep
it somewhere else?

On Wed, Nov 24, 2010 at 6:39 PM,  <antocuni at codespeak.net> wrote:
> Author: antocuni
> Date: Wed Nov 24 17:39:03 2010
> New Revision: 79480
>
> Added:
> ? pypy/trunk/pypy/jit/tool/cpython.vmrss
> Log:
> add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456
>
>
>
> Added: pypy/trunk/pypy/jit/tool/cpython.vmrss
> ==============================================================================
> --- (empty file)
> +++ pypy/trunk/pypy/jit/tool/cpython.vmrss ? ? ?Wed Nov 24 17:39:03 2010
> @@ -0,0 +1,4101 @@
> +124
> +16600
> +20620
> +20640
> +22036
> +94084
> +94084
> +94108
> +94108
> +94108
> +94108
> +94108
> +94120
> +94164
> +94160
> +94160
> +94160
> +94160
> +94160
> +94216
> +110644
> +123144
> +135236
> +143680
> +148500
> +153104
> +157088
> +160640
> +164760
> +167992
> +163108
> +163232
> +163232
> +163436
> +163436
> +163436
> +163444
> +163444
> +163444
> +163448
> +163448
> +163448
> +163448
> +163448
> +163448
> +167060
> +170948
> +174432
> +177212
> +177216
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176508
> +176520
> +176520
> +176520
> +176520
> +176520
> +176520
> +176520
> +176520
> +176520
> +176540
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +176544
> +179120
> +187120
> +189380
> +191052
> +192156
> +193320
> +194900
> +195860
> +198516
> +201484
> +202600
> +202600
> +202600
> +202600
> +202832
> +202832
> +202836
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +202840
> +207784
> +212136
> +216320
> +220508
> +224696
> +228876
> +245088
> +245088
> +247844
> +252032
> +256212
> +260400
> +264592
> +268776
> +272776
> +275060
> +279244
> +283428
> +287616
> +291032
> +293900
> +298080
> +302272
> +304364
> +308548
> +310644
> +312740
> +316924
> +319016
> +323208
> +325296
> +327392
> +331572
> +333668
> +335760
> +337856
> +354328
> +356424
> +358520
> +362700
> +364792
> +366892
> +368984
> +371080
> +373168
> +375260
> +377356
> +379448
> +381540
> +383636
> +383636
> +385732
> +387820
> +390032
> +391160
> +392292
> +394552
> +396816
> +397092
> +399072
> +401340
> +403600
> +405860
> +408008
> +408148
> +412640
> +414900
> +417164
> +419420
> +421680
> +423944
> +426204
> +428464
> +430724
> +432768
> +434980
> +436476
> +437932
> +440332
> +441984
> +442740
> +445152
> +447688
> +449148
> +449960
> +452436
> +454712
> +454896
> +457180
> +458888
> +459688
> +462040
> +463480
> +464408
> +466812
> +467244
> +469224
> +471096
> +471684
> +474136
> +474328
> +476508
> +478872
> +481356
> +483472
> +483780
> +486072
> +488480
> +489668
> +490888
> +493420
> +495704
> +496836
> +498116
> +500520
> +502756
> +503668
> +505400
> +507760
> +509296
> +510204
> +512764
> +514708
> +515508
> +517372
> +519764
> +520648
> +522188
> +524596
> +525524
> +527004
> +529412
> +534224
> +536632
> +538080
> +539216
> +541588
> +542560
> +543384
> +543384
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +518804
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519024
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519028
> +519032
> +519032
> +519032
> +519032
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519036
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519048
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519052
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519056
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519060
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519064
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +519068
> +520592
> +520592
> +520904
> +522740
> +522740
> +523212
> +525256
> +525256
> +525316
> +526552
> +526552
> +526560
> +528508
> +528508
> +528508
> +530040
> +530040
> +530040
> +532684
> +532684
> +532684
> +534948
> +534948
> +534948
> +538028
> +538028
> +538028
> +541404
> +541448
> +541448
> +543784
> +543784
> +543784
> +545184
> +545476
> +545768
> +545832
> +545832
> +545832
> +546016
> +546016
> +546128
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546184
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546228
> +546708
> +546708
> +546708
> +547988
> +550420
> +550420
> +550420
> +552896
> +555796
> +555796
> +555796
> +559136
> +560280
> +560280
> +560996
> +562504
> +563772
> +564672
> +564672
> +565268
> +567936
> +568884
> +569028
> +569236
> +569292
> +569292
> +570236
> +572960
> +573980
> +573980
> +574508
> +577404
> +579188
> +579188
> +579508
> +582836
> +584468
> +584468
> +585544
> +591292
> +591292
> +591292
> +595868
> +597588
> +597588
> +598404
> +602772
> +603964
> +603964
> +605488
> +609740
> +610468
> +610468
> +611884
> +616440
> +617108
> +617108
> +618156
> +622276
> +623784
> +623784
> +624128
> +624376
> +625544
> +626736
> +627112
> +627112
> +627116
> +627656
> +628836
> +628836
> +628836
> +629160
> +630180
> +630488
> +630488
> +630492
> +631288
> +632144
> +632144
> +632144
> +632172
> +632688
> +633220
> +633220
> +633220
> +633284
> +633756
> +633916
> +633916
> +633916
> +634012
> +634608
> +634608
> +634608
> +634624
> +634732
> +635144
> +635144
> +635144
> +635196
> +635680
> +635868
> +635868
> +635944
> +638440
> +639964
> +639980
> +639980
> +640056
> +641052
> +642064
> +642064
> +642064
> +642248
> +643080
> +643832
> +643832
> +643832
> +644116
> +646500
> +647424
> +648236
> +649032
> +649156
> +649156
> +649256
> +651556
> +652056
> +652504
> +652860
> +653168
> +653440
> +653876
> +654096
> +654304
> +654304
> +654304
> +654756
> +655648
> +657064
> +657064
> +657064
> +657064
> +657488
> +657756
> +657756
> +657756
> +658112
> +658736
> +658736
> +658736
> +658796
> +659304
> +659376
> +659376
> +659376
> +659848
> +661000
> +661172
> +661172
> +661236
> +662240
> +663240
> +663508
> +663508
> +663660
> +664324
> +665512
> +665764
> +665764
> +665764
> +666448
> +667692
> +668112
> +668112
> +668112
> +668624
> +669508
> +670388
> +670536
> +670536
> +670536
> +670564
> +671288
> +672268
> +672268
> +672268
> +672676
> +674504
> +675072
> +675072
> +675072
> +675156
> +675896
> +676700
> +676700
> +676700
> +676820
> +677988
> +678628
> +678628
> +678628
> +678776
> +679744
> +680400
> +680400
> +680400
> +705092
> +705156
> +705156
> +705156
> +705156
> +705156
> +705156
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705160
> +705264
> +705264
> +705264
> +705400
> +705476
> +706168
> +706292
> +706292
> +706292
> +706504
> +706568
> +706980
> +707012
> +707012
> +707012
> +707196
> +707280
> +707904
> +707904
> +707904
> +707924
> +708112
> +708176
> +708676
> +708676
> +708676
> +708696
> +708892
> +708984
> +709588
> +709588
> +709588
> +709612
> +709804
> +709848
> +710300
> +710300
> +710300
> +710676
> +710520
> +710604
> +711156
> +711156
> +711156
> +711336
> +711352
> +711576
> +712080
> +712080
> +712080
> +712300
> +712408
> +712500
> +712648
> +712648
> +712648
> +712648
> +713060
> +713300
> +713716
> +713716
> +713716
> +714072
> +714196
> +714568
> +714568
> +714568
> +714596
> +714956
> +715112
> +715808
> +715808
> +715808
> +717504
> +717628
> +717660
> +717660
> +717660
> +718620
> +719048
> +719424
> +719424
> +719424
> +719480
> +719924
> +720612
> +720612
> +720612
> +720620
> +722584
> +722848
> +722848
> +722848
> +723060
> +724108
> +724604
> +724604
> +724604
> +725108
> +726168
> +726348
> +726348
> +726348
> +727216
> +728204
> +728204
> +728204
> +728324
> +729396
> +730152
> +730152
> +730152
> +730396
> +736796
> +736800
> +736800
> +736800
> +736800
> +736800
> +736800
> +736800
> +736800
> +737136
> +738296
> +738400
> +738400
> +738400
> +739092
> +740128
> +740128
> +740128
> +740140
> +741092
> +741980
> +741980
> +741980
> +742060
> +743060
> +743796
> +743796
> +743796
> +743836
> +744440
> +745348
> +745348
> +745348
> +745400
> +746108
> +746848
> +746952
> +746952
> +746952
> +747496
> +748608
> +748744
> +749084
> +749084
> +749084
> +749172
> +750172
> +751468
> +751592
> +751592
> +751592
> +751688
> +751928
> +752152
> +752308
> +759152
> +760152
> +760152
> +760152
> +756356
> +754816
> +756356
> +756356
> +756356
> +756688
> +756688
> +756820
> +757152
> +757200
> +757400
> +757432
> +757432
> +757432
> +757632
> +757956
> +758404
> +758844
> +759480
> +760064
> +760552
> +760552
> +760552
> +760560
> +761180
> +761632
> +762288
> +762800
> +763700
> +764504
> +764716
> +764716
> +764716
> +764940
> +765388
> +765936
> +767748
> +767056
> +767300
> +767484
> +767868
> +768316
> +768316
> +768316
> +768316
> +768700
> +768828
> +768700
> +769340
> +769260
> +771008
> +771552
> +771652
> +771716
> +772580
> +772708
> +772740
> +772740
> +772740
> +772292
> +772740
> +772944
> +773188
> +773700
> +774084
> +774084
> +774404
> +774020
> +774532
> +774020
> +774596
> +774340
> +774468
> +774468
> +774468
> +774724
> +774792
> +774980
> +775368
> +775816
> +775816
> +776264
> +777480
> +778292
> +778408
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778440
> +778632
> +778696
> +778952
> +779016
> +779016
> +779016
> +779016
> +780812
> +780812
> +780812
> +781068
> +781580
> +781772
> +781712
> +781868
> +782092
> +782420
> +782796
> +782796
> +782796
> +782796
> +782668
> +783128
> +783436
> +783820
> +784076
> +784332
> +784908
> +785164
> +785500
> +786188
> +786188
> +786188
> +786188
> +786188
> +786188
> +786188
> +786412
> +786896
> +787084
> +787404
> +787532
> +787724
> +787568
> +788108
> +788428
> +788748
> +788752
> +789520
> +789520
> +789520
> +788880
> +789440
> +789452
> +789516
> +790092
> +790284
> +790604
> +790860
> +791052
> +791372
> +791628
> +792012
> +792012
> +792396
> +792780
> +792780
> +792780
> +792780
> +792780
> +793228
> +793228
> +793868
> +793868
> +793996
> +793996
> +794636
> +794636
> +794636
> +795084
> +795084
> +795468
> +795532
> +795980
> +795980
> +796300
> +796364
> +796364
> +796364
> +796364
> +796748
> +797132
> +797132
> +797516
> +797644
> +797644
> +798380
> +798604
> +798860
> +798924
> +799372
> +799564
> +799756
> +799756
> +799756
> +799756
> +799816
> +800292
> +800792
> +801312
> +801816
> +802364
> +802880
> +803456
> +803660
> +803660
> +803660
> +803812
> +804388
> +804960
> +805516
> +806084
> +806668
> +807324
> +807980
> +807980
> +807980
> +807980
> +808416
> +809084
> +809784
> +810492
> +811160
> +812900
> +813476
> +813876
> +813876
> +813876
> +813876
> +814508
> +815152
> +815864
> +816556
> +817260
> +817916
> +818708
> +819272
> +819352
> +819352
> +819352
> +819352
> +819884
> +820308
> +820888
> +821012
> +821012
> +821204
> +821588
> +821588
> +821588
> +821972
> +822228
> +822164
> +822932
> +823252
> +823252
> +823252
> +823252
> +823252
> +823256
> +823612
> +823920
> +823936
> +824140
> +824168
> +824220
> +824280
> +824400
> +824664
> +825776
> +825776
> +825776
> +825776
> +832168
> +832168
> +832168
> +832168
> +832808
> +833492
> +833492
> +833492
> +833492
> +833700
> +834308
> +834428
> +834428
> +834428
> +834780
> +836216
> +836216
> +836216
> +836836
> +839308
> +839308
> +839308
> +839308
> +839416
> +840344
> +840344
> +840344
> +840344
> +841724
> +841724
> +841724
> +841724
> +843800
> +843800
> +843800
> +843800
> +845692
> +846072
> +846072
> +846072
> +846096
> +846736
> +847096
> +847156
> +847156
> +847156
> +847160
> +847180
> +847360
> +847360
> +847360
> +847360
> +847416
> +849248
> +849248
> +849248
> +849248
> +849716
> +849716
> +849716
> +849716
> +849740
> +851168
> +851168
> +851168
> +851168
> +854544
> +854544
> +854544
> +854544
> +854564
> +855332
> +855332
> +855332
> +855332
> +857092
> +857092
> +857092
> +857092
> +858000
> +859388
> +859388
> +859388
> +859388
> +861584
> +861584
> +861584
> +861584
> +861584
> +863464
> +863464
> +863464
> +863464
> +864304
> +866500
> +866500
> +866500
> +866500
> +868952
> +868952
> +868952
> +868952
> +869740
> +872108
> +872108
> +872108
> +872108
> +873208
> +875564
> +875564
> +875564
> +875564
> +878568
> +878568
> +878568
> +878568
> +878576
> +880780
> +880780
> +880780
> +880780
> +884048
> +884048
> +884048
> +884048
> +884048
> +884048
> +886516
> +886516
> +886516
> +886516
> +886516
> +886536
> +888112
> +888112
> +888112
> +888112
> +888112
> +888112
> +888656
> +888984
> +888984
> +888984
> +888984
> +888984
> +888984
> +889076
> +890288
> +890288
> +890288
> +890288
> +890288
> +890288
> +890404
> +892900
> +892900
> +892900
> +892900
> +892900
> +892900
> +892900
> +892900
> +895760
> +895760
> +895760
> +895760
> +895760
> +895760
> +895920
> +897624
> +897624
> +897624
> +897624
> +897624
> +897624
> +897624
> +897624
> +897628
> +898024
> +898024
> +898024
> +898024
> +898024
> +898060
> +900024
> +900024
> +900024
> +900024
> +900024
> +900024
> +900240
> +901436
> +901436
> +901436
> +901436
> +901436
> +901556
> +903116
> +903116
> +903116
> +903116
> +903116
> +903128
> +905084
> +905084
> +905084
> +905084
> +905084
> +905084
> +905096
> +906832
> +906832
> +906832
> +906832
> +906832
> +906832
> +908916
> +908916
> +908916
> +908916
> +908916
> +908916
> +908916
> +910720
> +910720
> +910720
> +910720
> +910720
> +910720
> +911780
> +912072
> +912072
> +912072
> +912072
> +914472
> +914472
> +914472
> +914472
> +914472
> +917120
> +917120
> +917120
> +917120
> +917120
> +919056
> +919056
> +919056
> +919056
> +919056
> +920316
> +920316
> +920316
> +920316
> +920316
> +920892
> +920892
> +920892
> +920892
> +920892
> +922996
> +922996
> +922996
> +922996
> +922996
> +925564
> +925564
> +925564
> +925564
> +925564
> +927780
> +927780
> +927780
> +927780
> +928016
> +928936
> +929048
> +930864
> +930864
> +930864
> +930864
> +932980
> +933304
> +933304
> +933304
> +933304
> +933540
> +934292
> +935452
> +935528
> +935528
> +935528
> +935528
> +935528
> +935528
> +935528
> +935528
> +935600
> +936112
> +936112
> +936208
> +936208
> +936208
> +936208
> +936308
> +936308
> +936316
> +936368
> +936368
> +936368
> +936368
> +936368
> +936368
> +936368
> +936368
> +936368
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +936376
> +937404
> +937404
> +937404
> +937404
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +939968
> +940516
> +940516
> +940516
> +940516
> +947168
> +947168
> +947168
> +947168
> +951948
> +951948
> +951948
> +951948
> +953488
> +956916
> +956916
> +956916
> +956916
> +952296
> +955376
> +953420
> +953260
> +953900
> +953516
> +953900
> +955052
> +953516
> +953516
> +954284
> +953324
> +953516
> +956208
> +956208
> +956208
> +956208
> +954668
> +948988
> +948988
> +948988
> +948988
> +948988
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +945908
> +947704
> +948068
> +948068
> +948068
> +948068
> +948068
> +948552
> +949024
> +949024
> +949024
> +949024
> +949024
> +949944
> +950492
> +950616
> +950616
> +950616
> +950616
> +951416
> +952000
> +952564
> +952564
> +952564
> +952564
> +952620
> +953296
> +953952
> +954332
> +954332
> +954332
> +954332
> +954532
> +955532
> +956216
> +956216
> +956216
> +956216
> +956216
> +956584
> +957396
> +957704
> +957704
> +957704
> +957704
> +957740
> +958620
> +959444
> +959444
> +959444
> +959444
> +959444
> +960224
> +963784
> +963868
> +963868
> +963868
> +963868
> +963872
> +964684
> +972452
> +972452
> +972452
> +972452
> +972452
> +972452
> +973220
> +974548
> +974548
> +974548
> +974548
> +974548
> +975540
> +977704
> +978792
> +978792
> +978792
> +978792
> +978792
> +981296
> +982700
> +983180
> +983180
> +983180
> +983180
> +983368
> +985060
> +987512
> +987688
> +987688
> +987688
> +987688
> +988180
> +990724
> +993084
> +993124
> +993124
> +993124
> +993124
> +993604
> +995444
> +996640
> +996640
> +996640
> +996640
> +996640
> +997892
> +999160
> +1001452
> +1001452
> +1001452
> +1001452
> +1001452
> +1002264
> +1004120
> +1004916
> +1004916
> +1004916
> +1004916
> +1004916
> +1006648
> +1010244
> +1010548
> +1010548
> +1010548
> +1010548
> +1010572
> +1011860
> +1013748
> +1017264
> +1017264
> +1017264
> +1017264
> +1017264
> +1019096
> +1021044
> +1021044
> +1021044
> +1021044
> +1021044
> +1021536
> +1023636
> +1024964
> +1024964
> +1024964
> +1024964
> +1024964
> +1025592
> +1027672
> +1029384
> +1029384
> +1029384
> +1029384
> +1029384
> +1030056
> +1032244
> +1033756
> +1033756
> +1033756
> +1033756
> +1033756
> +1034384
> +1035856
> +1036588
> +1038428
> +1038428
> +1038428
> +1038428
> +1038428
> +1039288
> +1041088
> +1042508
> +1042508
> +1042508
> +1042508
> +1042508
> +1043544
> +1045280
> +1046580
> +1046580
> +1046580
> +1046580
> +1046580
> +1047040
> +1048560
> +1049872
> +1050576
> +1050576
> +1050576
> +1050576
> +1050576
> +1052016
> +1056044
> +1057360
> +1057360
> +1057360
> +1057360
> +1057360
> +1058336
> +1059900
> +1061244
> +1061704
> +1061704
> +1061704
> +1061704
> +1061704
> +1063148
> +1063972
> +1065404
> +1067064
> +1067536
> +1067536
> +1067536
> +1067536
> +1067536
> +1069284
> +1070524
> +1072340
> +1072836
> +1072836
> +1072836
> +1072836
> +1072836
> +1073584
> +1074592
> +1076404
> +1076832
> +1076832
> +1076832
> +1076832
> +1076832
> +1078124
> +1079640
> +1080220
> +1080644
> +1080736
> +1080736
> +1080736
> +1080736
> +1080924
> +1082868
> +1083368
> +1084412
> +1084412
> +1084412
> +1084412
> +1084412
> +1085216
> +1087068
> +1087960
> +1087960
> +1087960
> +1087960
> +1087960
> +1089788
> +1093132
> +1095064
> +1095064
> +1095064
> +1095064
> +1095064
> +1095140
> +1097092
> +1098948
> +1098948
> +1098948
> +1098948
> +1098948
> +1098980
> +1100812
> +1102032
> +1102032
> +1102032
> +1102032
> +1102032
> +1105464
> +1116944
> +1119248
> +1120364
> +1120364
> +1120364
> +1120364
> +1120364
> +1121568
> +1122908
> +1123680
> +1124604
> +1125076
> +1125076
> +1125076
> +1125076
> +1125076
> +1126292
> +1128160
> +1129952
> +1129952
> +1129952
> +1129952
> +1129952
> +1130496
> +1131884
> +1133032
> +1134204
> +1135460
> +1135636
> +1135636
> +1135636
> +1135636
> +1135636
> +1136764
> +1138048
> +1139412
> +1140764
> +1141164
> +1141188
> +1142440
> +1142440
> +1142440
> +1142440
> +1142440
> +1142440
> +1143980
> +1145876
> +1146576
> +1146576
> +1146576
> +1146576
> +1146576
> +1146576
> +1147680
> +1148328
> +1148960
> +1148960
> +1148960
> +1148960
> +1148960
> +1149004
> +1150700
> +1152228
> +1153364
> +1153364
> +1153520
> +1153784
> +1154588
> +1154680
> +1154712
> +1154728
> +1154784
> +1154992
> +1155356
> +1155620
> +1155856
> +1156044
> +1156420
> +1157392
> +1158760
> +1158980
> +1158988
> +1159000
> +1162724
> +1162740
> +1162788
> +1163112
> +1163188
> +1163188
> +1163188
> +1163188
> +1163188
> +1163384
> +1165668
> +1166648
> +1166652
> +1166664
> +1166676
> +1166688
> +1166692
> +1166696
> +1166700
> +1166700
> +1172848
> +1172852
> +1174888
> +1176824
> +1176836
> +1176852
> +1176860
> +1176876
> +1176880
> +1176888
> +1176892
> +1176900
> +1176912
> +1176944
> +1177248
> +1177712
> +1178172
> +1178536
> +1178656
> +1178780
> +1178920
> +1179044
> +1179188
> +1179384
> +1180296
> +1180300
> +1180300
> +1180300
> +1180300
> +1180300
> +1180300
> +1180372
> +1180380
> +1180468
> +1180524
> +1180524
> +1180524
> +1180524
> +1180524
> +1180576
> +1180580
> +1180644
> +1180684
> +1180684
> +1180684
> +1180684
> +1180684
> +1180684
> +1180724
> +1180756
> +1180852
> +1180904
> +1180904
> +1180904
> +1180904
> +1180904
> +1180904
> +1181096
> +1181400
> +1181744
> +1181744
> +1181744
> +1181744
> +1181744
> +1181744
> +1181936
> +1181936
> +1181936
> +1181936
> +1181936
> +1181936
> +1181936
> +1181936
> +1181936
> +1181972
> +1182004
> +1182004
> +1182004
> +1182004
> +1182004
> +1182004
> +1182004
> +1182004
> +1182004
> +1182128
> +1182156
> +1182156
> +1182156
> +1182156
> +1182156
> +1182156
> +1182156
> +1182156
> +1182180
> +1182180
> +1182180
> +1182180
> +1182180
> +1182180
> +1182180
> +1182368
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1182516
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183576
> +1183596
> +1183772
> +1183772
> +1183772
> +1183772
> +1183772
> +1183772
> +1183772
> +1183772
> +1183772
> +1183776
> +1183776
> +1183776
> +1183776
> +1183776
> +1183776
> +1183776
> +1183776
> +1183776
> +1183952
> +1184000
> +1184000
> +1184000
> +1184000
> +1184000
> +1184000
> +1184000
> +1184200
> +1184832
> +1184832
> +1184832
> +1184832
> +1184832
> +1184832
> +1184928
> +1184928
> +1184940
> +1184940
> +1184940
> +1184940
> +1184940
> +1184940
> +1184940
> +1184940
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185144
> +1185196
> +1185196
> +1185196
> +1185196
> +1185196
> +1185196
> +1185196
> +1185196
> +1185196
> +1185268
> +1185268
> +1185268
> +1185268
> +1185268
> +1185268
> +1185268
> +1185268
> +1186444
> +1186776
> +1186776
> +1186776
> +1186776
> +1186776
> +1187664
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188072
> +1188168
> +1188168
> +1188168
> +1188168
> +1188168
> +1188168
> +1188168
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189240
> +1189292
> +1189292
> +1189292
> +1189292
> +1189292
> +1189292
> +1189292
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189532
> +1189704
> +1189708
> +1189708
> +1189708
> +1189708
> +1189708
> +1190160
> +1190160
> +1190160
> +1190160
> +1190160
> +1190160
> +1190160
> +1190160
> +1190160
> +1191100
> +1191100
> +1191100
> +1191100
> +1191100
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191316
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191748
> +1191772
> +1191772
> +1191772
> +1191772
> +1191772
> +1191772
> +1191772
> +1191772
> +1192964
> +1192964
> +1192964
> +1192964
> +1192964
> +1192964
> +1192964
> +1193060
> +1193060
> +1193060
> +1193060
> +1193060
> +1193060
> +1193060
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193072
> +1193076
> +1193076
> +1193076
> +1193076
> +1193076
> +1193076
> +1193124
> +1193124
> +1193360
> +1194108
> +1194108
> +1194108
> +1194108
> +1194108
> +1193380
> +1193460
> +1193460
> +1193460
> +1193460
> +1193460
> +1193460
> +1193460
> +1193460
> +1193792
> +1193792
> +1193792
> +1193792
> +1193792
> +1193792
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194000
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194048
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194328
> +1194360
> +1194360
> +1194360
> +1194360
> +1194360
> +1194360
> +1194360
> +1194508
> +1194508
> +1194508
> +1194508
> +1194508
> +1194508
> +1194512
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194668
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1194912
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196596
> +1196660
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196764
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1196948
> +1197236
> +1197236
> +1197236
> +1197236
> +1197236
> +1197236
> +1197236
> +1197236
> +1197236
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197288
> +1197376
> +1197384
> +1197596
> +1197596
> +1197596
> +1197596
> +1197596
> +1197596
> +1197596
> +1197596
> +1197588
> +1197588
> +1197588
> +1197588
> +1197588
> +1197588
> +1197588
> +1197588
> +1197564
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197460
> +1197492
> +1197508
> +1197516
> +1197516
> +1197516
> +1197516
> +1197516
> +1197516
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>


From anto.cuni at gmail.com  Thu Nov 25 08:45:52 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 25 Nov 2010 08:45:52 +0100
Subject: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
In-Reply-To: <AANLkTikvrsprwj9ehh8ZRndDvqKCYY7zms4um6uEqHCc@mail.gmail.com>
References: <20101124163904.C308C50810@codespeak.net>
	<AANLkTikvrsprwj9ehh8ZRndDvqKCYY7zms4um6uEqHCc@mail.gmail.com>
Message-ID: <4CEE1430.2@gmail.com>

On 25/11/10 07:40, Maciej Fijalkowski wrote:
> Is pypy repository really a place for such files? Maybe we should keep
> it somewhere else?

well, it's the memory usage of cpython when translating pypy. Why the pypy 
repo should not be the place for a pypy-related file?

> On Wed, Nov 24, 2010 at 6:39 PM,<antocuni at codespeak.net>  wrote:
>> Author: antocuni
>> Date: Wed Nov 24 17:39:03 2010
>> New Revision: 79480
>>
>> Added:
>>    pypy/trunk/pypy/jit/tool/cpython.vmrss
>> Log:
>> add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456



From anto.cuni at gmail.com  Thu Nov 25 08:53:11 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 25 Nov 2010 08:53:11 +0100
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
In-Reply-To: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
Message-ID: <4CEE15E7.3030409@gmail.com>

Hi Wim,

On 25/11/10 01:32, wlavrijsen at lbl.gov wrote:
> Hi,
>
> so I'm revising my numbers after finding out that I was using a debug version
> of ROOT/Reflex ...
>
> PyROOT:   48.6
> PyCintex: 50.2
> pypy-c:    5.5
> C++:       0.05
>

wow, impressive numbers :-).

You said that the benchmark you used is a loop calling a c++ function that 
does nothing. What is the signature of that function?
If you remember, in interp_cppyy there is a fast path that gives huge speedups 
when calling a c++ function which takes an integer and returns an integer, so 
depending on the signature you get very different numbers today.

The good news is that nowadays the JIT has got the capability to optimise 
external calls in a general way: this means that as soon as we integrate it 
with cppyy we can have all calls as fast as the ones that go through the 
current fast path.

ciao,
Anto


From fijall at gmail.com  Thu Nov 25 09:24:13 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Nov 2010 10:24:13 +0200
Subject: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
In-Reply-To: <4CEE1430.2@gmail.com>
References: <20101124163904.C308C50810@codespeak.net>
	<AANLkTikvrsprwj9ehh8ZRndDvqKCYY7zms4um6uEqHCc@mail.gmail.com>
	<4CEE1430.2@gmail.com>
Message-ID: <AANLkTinWQoPgs3pk-4HeCaF0WjU9vhKE8unbomvJRtyZ@mail.gmail.com>

On Thu, Nov 25, 2010 at 9:45 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On 25/11/10 07:40, Maciej Fijalkowski wrote:
>>
>> Is pypy repository really a place for such files? Maybe we should keep
>> it somewhere else?
>
> well, it's the memory usage of cpython when translating pypy. Why the pypy
> repo should not be the place for a pypy-related file?

For the same reason why we don't put extradoc in checkout I guess.

>
>> On Wed, Nov 24, 2010 at 6:39 PM,<antocuni at codespeak.net> ?wrote:
>>>
>>> Author: antocuni
>>> Date: Wed Nov 24 17:39:03 2010
>>> New Revision: 79480
>>>
>>> Added:
>>> ? pypy/trunk/pypy/jit/tool/cpython.vmrss
>>> Log:
>>> add the vmrss file produced by armin's memusage.py when running
>>> ./translate.py -Ojit at rev 79456
>
>


From arigo at tunes.org  Thu Nov 25 11:18:39 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Nov 2010 11:18:39 +0100
Subject: [pypy-dev] [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool
In-Reply-To: <AANLkTinWQoPgs3pk-4HeCaF0WjU9vhKE8unbomvJRtyZ@mail.gmail.com>
References: <20101124163904.C308C50810@codespeak.net>
	<AANLkTikvrsprwj9ehh8ZRndDvqKCYY7zms4um6uEqHCc@mail.gmail.com>
	<4CEE1430.2@gmail.com>
	<AANLkTinWQoPgs3pk-4HeCaF0WjU9vhKE8unbomvJRtyZ@mail.gmail.com>
Message-ID: <AANLkTimjHqQKosR1Ekgn8xsxw4V3uaV3z8Tr=8ztvdxc@mail.gmail.com>

Hi Anto,

On Thu, Nov 25, 2010 at 9:24 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> For the same reason why we don't put extradoc in checkout I guess.

I tend to agree with fijal that this file should be in extradoc/somepath...


Armin


From wlavrijsen at lbl.gov  Thu Nov 25 17:46:48 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Thu, 25 Nov 2010 08:46:48 -0800 (PST)
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
In-Reply-To: <4CEE15E7.3030409@gmail.com>
References: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
	<4CEE15E7.3030409@gmail.com>
Message-ID: <alpine.LNX.2.00.1011250841320.28632@pc3-wlav.lbl.gov>

Hi Anto,

> You said that the benchmark you used is a loop calling a c++ function that 
> does nothing. What is the signature of that function?
> If you remember, in interp_cppyy there is a fast path that gives huge 
> speedups when calling a c++ function which takes an integer and returns an 
> integer, so depending on the signature you get very different numbers today.

yes, but the fast path was disabled (commented out) when updating the branch
with trunk (I needed a few casts in the JIT that were in trunk already, but
not in the branch). It should thus be in addition when it's working again.
(And to be sure, using a float argument makes no difference for the numbers.)

Myself, I'm just working on functionality right now. A factor of almost 10
is already good enough to start working on a functional demo of real code,
given that most of our analysis is I/O bound.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From holger at merlinux.eu  Thu Nov 25 20:02:05 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 25 Nov 2010 20:02:05 +0100
Subject: [pypy-dev] why pypy startup is currently ~5 times slower than
	python?
Message-ID: <20101125190205.GA1009@trillke.net>

Hi,

using the following script:

    import sys, subprocess

    for i in range(1000):
        subprocess.call([sys.executable, "--version"])

i get these results repeatedly:

(0)hpk at teta:~/p/pytest$ time python2.7 r.py 2>/dev/null >/dev/null
real    0m1.885s
user    0m0.110s
sys     0m0.480s

(0)hpk at teta:~/p/pytest$ time pypy r.py 2>/dev/null >/dev/null
real    0m8.763s
user    0m1.730s
sys     0m2.880s

IOW, pypy seems to be consistently 5-6 times slower on startup.
Does anybody have explanations why?   When we did measurements
for the Nokia Maemo device pypy was actually much faster IIRC.

holger


From holger at merlinux.eu  Thu Nov 25 20:21:03 2010
From: holger at merlinux.eu (holger krekel)
Date: Thu, 25 Nov 2010 20:21:03 +0100
Subject: [pypy-dev] faster actually (was Re: why pypy startup is
	currently	~5 times slower than python?)
In-Reply-To: <20101125190205.GA1009@trillke.net>
References: <20101125190205.GA1009@trillke.net>
Message-ID: <20101125192103.GB1009@trillke.net>

Well, it seems to be the version-printing.  Using 

    subprocess.call([sys.executable, "-c", "pass"])

or "print 42" or other simple commands pypy seems
faster up to 2 times. Not sure care is needed for
speed of --version :)

cheers,
holger

On Thu, Nov 25, 2010 at 20:02 +0100, holger krekel wrote:
> using the following script:
> 
>     import sys, subprocess
> 
>     for i in range(1000):
>         subprocess.call([sys.executable, "--version"])
> 
> i get these results repeatedly:
> 
> (0)hpk at teta:~/p/pytest$ time python2.7 r.py 2>/dev/null >/dev/null
> real    0m1.885s
> user    0m0.110s
> sys     0m0.480s
> 
> (0)hpk at teta:~/p/pytest$ time pypy r.py 2>/dev/null >/dev/null
> real    0m8.763s
> user    0m1.730s
> sys     0m2.880s
> 
> IOW, pypy seems to be consistently 5-6 times slower on startup.
> Does anybody have explanations why?   When we did measurements
> for the Nokia Maemo device pypy was actually much faster IIRC.
> 
> holger
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 


From anto.cuni at gmail.com  Fri Nov 26 11:48:49 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 26 Nov 2010 11:48:49 +0100
Subject: [pypy-dev] ype object 'BlackholeInterpreter' has no attribute
 'bhimpl_direct_ptradd'
In-Reply-To: <alpine.LNX.2.00.1011250841320.28632@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
	<4CEE15E7.3030409@gmail.com>
	<alpine.LNX.2.00.1011250841320.28632@pc3-wlav.lbl.gov>
Message-ID: <4CEF9091.7010405@gmail.com>

On 25/11/10 17:46, wlavrijsen at lbl.gov wrote:

> yes, but the fast path was disabled (commented out) when updating the branch
> with trunk (I needed a few casts in the JIT that were in trunk already, but
> not in the branch). It should thus be in addition when it's working again.
> (And to be sure, using a float argument makes no difference for the numbers.)
>
> Myself, I'm just working on functionality right now. A factor of almost 10
> is already good enough to start working on a functional demo of real code,
> given that most of our analysis is I/O bound.

wow, if you did not use the fast path, then the 10x factor it's even more 
impressive :-).

I think that yesterday Carl Friedrich added support to cppyy for calling the 
function through the new JIT-friendly "rlib.libffi" module, so now you should 
get the "fast-path" performance for most of the calls.  Carl, could you post 
the benchmarks you did please?

ciao,
Anto


From cfbolz at gmx.de  Fri Nov 26 16:31:44 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 26 Nov 2010 16:31:44 +0100
Subject: [pypy-dev] cppyy performance numbers (was: ype object
 'BlackholeInterpreter' has no attribute 'bhimpl_direct_ptradd')
In-Reply-To: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
Message-ID: <4CEFD2E0.8050201@gmx.de>

Hi Wim, hi all,

On 11/25/2010 01:32 AM, wlavrijsen at lbl.gov wrote:
> so I'm revising my numbers after finding out that I was using a debug
> version
> of ROOT/Reflex ...
>
> PyROOT: 48.6
> PyCintex: 50.2
> pypy-c: 5.5
> C++: 0.05

I did some benchmarks with the new fast path that I just added, it gives 
the following numbers:

pypy-c without fast path: 5.9 (my laptop seems a bit slower than yours)
pypy-c with fast path: 0.22
C++: 0.05

So Anto's and my optimization helped a lot, we got another factor of 25 
faster. A bit more than a factor of 4 slower than C++, and it should 
work for many (non-static) method and constructor calls.

>
> Note that although PyROOT speeds up only marginally in opt mode, pypy-c has
> a huge speedup. For PyROOT, this is b/c most of the time is spent in
> CPython
> through API calls and that was already in opt mode. For pypy-c, this is
> telling me quite clearly that I've got to work some on capi.py. ;)
>
> Of course, the factor 6 that I saw originally (v.s. 2 during the sprint) is
> now a factor 10 almost ...

Are you sure that the units are the same? Because if yes, it is a factor 
of 100, not 10.

Cheers,

Carl Friedrich

P.S.: For the people missing context, see here:
http://morepypy.blogspot.com/2010/07/cern-sprint-report-wrapping-c-libraries.html


From fijall at gmail.com  Fri Nov 26 19:23:45 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 26 Nov 2010 20:23:45 +0200
Subject: [pypy-dev] PyPy 1.4 released
Message-ID: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>

===============================
PyPy 1.4: Ouroboros in practice
===============================

We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough
in our long journey, as PyPy 1.4 is the first PyPy release that can translate
itself faster than CPython.  Starting today, we are using PyPy more for
our every-day development.  So may you :) You can download it here:

    http://pypy.org/download.html

What is PyPy
============

PyPy is a very compliant Python interpreter, almost a drop-in replacement
for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)

Among its new features, this release includes numerous performance improvements
(which made fast self-hosting possible), a 64-bit JIT backend, as well
as serious stabilization. As of now, we can consider the 32-bit and 64-bit
linux versions of PyPy stable enough to run `in production`_.

Numerous speed achievements are described on `our blog`_. Normalized speed
charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_
are available on benchmark website. For the impatient: yes, we got a lot faster!

More highlights
===============

* PyPy's built-in Just-in-Time compiler is fully transparent and
  automatically generated; it now also has very reasonable memory
  requirements.  The total memory used by a very complex and
  long-running process (translating PyPy itself) is within 1.5x to
  at most 2x the memory needed by CPython, for a speed-up of 2x.

* More compact instances.  All instances are as compact as if
  they had ``__slots__``.  This can give programs a big gain in
  memory.  (In the example of translation above, we already have
  carefully placed ``__slots__``, so there is no extra win.)

* `Virtualenv support`_: now PyPy is fully compatible with
virtualenv_: note that
  to use it, you need a recent version of virtualenv (>= 1.5).

* Faster (and JITted) regular expressions - huge boost in speeding up
  the `re` module.

* Other speed improvements, like JITted calls to functions like map().

.. _virtualenv: http://pypi.python.org/pypi/virtualenv
.. _`Virtualenv support`:
http://morepypy.blogspot.com/2010/08/using-virtualenv-with-pypy.html
.. _`in production`:
http://morepypy.blogspot.com/2010/11/running-large-radio-telescope-software.html
.. _`our blog`: http://morepypy.blogspot.com
.. _`pypy 1.4 and pypy 1.3`:
http://speed.pypy.org/comparison/?exe=1%2B41,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=1%2B41&chart=normal+bars
.. _`pypy 1.4 and cpython 2.6`:
http://speed.pypy.org/comparison/?exe=2%2B35,1%2B172&ben=1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20&env=1&hor=false&bas=2%2B35&chart=normal+bars

Cheers,

Carl Friedrich Bolz, Antonio Cuni, Maciej Fijalkowski,
Amaury Forgeot d'Arc, Armin Rigo and the PyPy team


From samuele.pedroni at gmail.com  Fri Nov 26 19:49:20 2010
From: samuele.pedroni at gmail.com (Samuele Pedroni)
Date: Fri, 26 Nov 2010 19:49:20 +0100
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
Message-ID: <AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>

On Fri, Nov 26, 2010 at 7:23 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> ===============================
> PyPy 1.4: Ouroboros in practice
> ===============================
>
> We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough
> in our long journey, as PyPy 1.4 is the first PyPy release that can translate
> itself faster than CPython. ?Starting today, we are using PyPy more for
> our every-day development. ?So may you :) You can download it here:
>
> ? ?http://pypy.org/download.html
>
> What is PyPy
> ============
>
> PyPy is a very compliant Python interpreter, almost a drop-in replacement
> for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)
>
> Among its new features, this release includes numerous performance improvements
> (which made fast self-hosting possible), a 64-bit JIT backend, as well
> as serious stabilization. As of now, we can consider the 32-bit and 64-bit
> linux versions of PyPy stable enough to run `in production`_.
>
> Numerous speed achievements are described on `our blog`_. Normalized speed
> charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython 2.6`_
> are available on benchmark website. For the impatient: yes, we got a lot faster!
>

congratulations!


From wlavrijsen at lbl.gov  Fri Nov 26 20:05:07 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Fri, 26 Nov 2010 11:05:07 -0800 (PST)
Subject: [pypy-dev] cppyy performance numbers (was: ype object
 'BlackholeInterpreter' has no attribute 'bhimpl_direct_ptradd')
In-Reply-To: <4CEFD2E0.8050201@gmx.de>
References: <alpine.LNX.2.00.1011241625120.15679@pc3-wlav.lbl.gov>
	<4CEFD2E0.8050201@gmx.de>
Message-ID: <alpine.LNX.2.00.1011261100040.13463@pc3-wlav.lbl.gov>

Hi Carl,

>> PyROOT: 48.6
>> PyCintex: 50.2
>> pypy-c: 5.5
>> C++: 0.05
>
> pypy-c without fast path: 5.9 (my laptop seems a bit slower than yours)
> pypy-c with fast path: 0.22
> C++: 0.05

in my case, with patched reflex, using the reflex-branch of pypy with the
libffi changes, I get a number of 0.41 for pypy-c.

So yes, that helped a lot, even when our numbers don't seem to line up, the
overall direction is definitely the way we want it. :) I'll see later when
moving up to trunk makes the numbers the same (on a relative basis at least).

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From fuzzyman at gmail.com  Fri Nov 26 23:15:11 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Fri, 26 Nov 2010 22:15:11 +0000
Subject: [pypy-dev] pypy 1.4 and virtualenv 1.5.1
Message-ID: <AANLkTi=SC_FY-HS-kkzTU4iB_-Mp+qf=rsgGDqf9R5Y8@mail.gmail.com>

Hello all,

Congratulations on the new release.

Unfortunately it seems that virtualenv 1.5.1 and pypy 1.4 do not play well
together on Mac OS X. At least the "--distribute" option doesn't work, which
is needed by tox.

I tried to create an issue on the PyPy issue tracker, but new registrants
are unable to create issues:

Here is the output on Linux:

$ virtualenv -p /compile/pypy-1.4/bin/pypy pypy --distribute
--no-site-packages
Running virtualenv with interpreter /compile/pypy-1.4/bin/pypy
New pypy-c executable in pypy/bin/pypy
Also creating executable in pypy/bin/pypy-c
Installing
distribute....................................................................................................................................................................................done.


Here is the failed output on OS X:

$ virtualenv -p /compile/pypy-1.4-osx/bin/pypy pypy --distribute
--no-site-packages
Running virtualenv with interpreter /compile/pypy-1.4-osx/bin/pypy
New pypy-c executable in pypy/bin/pypy
Also creating executable in pypy/bin/pypy-c
Please make sure you remove any previous custom paths from your
/Volumes/Second Drive/michael/.pydistutils.cfg file.
Installing distribute................................done.
Complete output from command /compile/pypy/bin/pypy
/compile/pypy/bin/easy_install /Library/Frameworks/Python.fra...ar.gz:
Traceback (most recent call last):
File "app_main.py", line 33, in run_toplevel
IOError: [Errno 2] No such file or directory: '/compile/pypy/bin/easy_install'
----------------------------------------
Traceback (most recent call last):
File "app_main.py", line 33, in run_toplevel
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
line 1647, in <module>
main()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
line 558, in main
prompt=options.prompt)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
line 656, in create_environment
install_pip(py_executable)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
line 415, in install_pip
filter_stdout=_filter_setup)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
line 624, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command /compile/pypy/bin/pypy /compile/pypy/bin/easy_install
/Library/Frameworks/Python.fra...ar.gz failed with error code 1


The nice thing is that for unittest2 and mock all tests pass with pypy 1.4.
Not unexpected, but still nice.

All the best,

Michael Foord

-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/d8f7588a/attachment.html>

From anto.cuni at gmail.com  Sat Nov 27 00:12:18 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 27 Nov 2010 00:12:18 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<amauryfa@gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>	<201011160222.oAG2Mrfv004177@theraft.openend.se>
	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
Message-ID: <4CF03ED2.3050801@gmail.com>

On 16/11/10 04:30, Dan Stromberg wrote:

> BTW, it might cause confusion down the road to call something that is
> basically like cpython's bsddb (Berkeley DB) by the name "dbm" in pypy's
> library.  In the cpython standard library, "dbm" is an interface to ndbm
> databases.  These all provide the same dictionary-like interface to Python
> programs, but have somewhat different API's to C, and pretty different,
> incompatible on-disk representations.

Hi Dan,
I played a bit (veeeery quickly) with dbm on both pypy and cpython, and I'm 
not sure I get what you mean when you say that our dbm.py is equivalent to 
cpython's bsddb. E.g., I can create a db on cpython and open it from pypy, so 
it seems that the two modules are compatible.

Moreover, I checked which libraries the links to. On CPython, it links to 
libdb-4.8.so:

viper2 ~ $ ldd /usr/lib/python2.6/lib-dynload/dbm.so
         linux-gate.so.1 =>  (0x00884000)
         libdb-4.8.so => /usr/lib/libdb-4.8.so (0x00110000)
         libpthread.so.0 => /lib/libpthread.so.0 (0x003de000)
         libc.so.6 => /lib/libc.so.6 (0x003f8000)
         /lib/ld-linux.so.2 (0x002e0000)

the pypy version first tries to open libdb.so, then libdb-4.5.so. I had to 
manually modify it to open version 4.8 (I agree that we should find a more 
general way to find it), but apart from that what I can see is that it uses 
the same underlying wrapper as CPython.

So, to summarise: could you elaborate a bit more why we should delete dbm.py 
from pypy?

ciao,
Anto


From angelflow at yahoo.com  Sat Nov 27 00:03:08 2010
From: angelflow at yahoo.com (Andy)
Date: Fri, 26 Nov 2010 15:03:08 -0800 (PST)
Subject: [pypy-dev] PyPy-JIT memory usage compared to CPython?
Message-ID: <160969.12027.qm@web111315.mail.gq1.yahoo.com>

Hi,

Congrats on the 1.4 release!

>From the benchmark results on your site, the performance looks great. I was wondering if you have any data showing relative memory usage for the various benchmark.

I'm particularly interested in the memory usage of PyPy-JIT vs. CPython when running Django. In general would PyPy-JIT increase or decrease memory usage?

Thanks
Andy


      


From holger at merlinux.eu  Sat Nov 27 00:19:41 2010
From: holger at merlinux.eu (holger krekel)
Date: Sat, 27 Nov 2010 00:19:41 +0100
Subject: [pypy-dev] pypy 1.4 and virtualenv 1.5.1
In-Reply-To: <AANLkTi=SC_FY-HS-kkzTU4iB_-Mp+qf=rsgGDqf9R5Y8@mail.gmail.com>
References: <AANLkTi=SC_FY-HS-kkzTU4iB_-Mp+qf=rsgGDqf9R5Y8@mail.gmail.com>
Message-ID: <20101126231940.GJ1009@trillke.net>

Hi Michael, 

On Fri, Nov 26, 2010 at 22:15 +0000, Michael Foord wrote:
> Congratulations on the new release.
> 
> Unfortunately it seems that virtualenv 1.5.1 and pypy 1.4 do not play well
> together on Mac OS X. At least the "--distribute" option doesn't work, which
> is needed by tox.

It's not needed, with tox you can configure "distribute=False" in your test env
and this would install setuptools.  

> I tried to create an issue on the PyPy issue tracker, but new registrants
> are unable to create issues:
> 
> Here is the output on Linux:
> 
> $ virtualenv -p /compile/pypy-1.4/bin/pypy pypy --distribute
> --no-site-packages

Does that work for you without --distribute? 

holger

> Running virtualenv with interpreter /compile/pypy-1.4/bin/pypy
> New pypy-c executable in pypy/bin/pypy
> Also creating executable in pypy/bin/pypy-c
> Installing
> distribute....................................................................................................................................................................................done.
> 
> 
> Here is the failed output on OS X:
> 
> $ virtualenv -p /compile/pypy-1.4-osx/bin/pypy pypy --distribute
> --no-site-packages
> Running virtualenv with interpreter /compile/pypy-1.4-osx/bin/pypy
> New pypy-c executable in pypy/bin/pypy
> Also creating executable in pypy/bin/pypy-c
> Please make sure you remove any previous custom paths from your
> /Volumes/Second Drive/michael/.pydistutils.cfg file.
> Installing distribute................................done.
> Complete output from command /compile/pypy/bin/pypy
> /compile/pypy/bin/easy_install /Library/Frameworks/Python.fra...ar.gz:
> Traceback (most recent call last):
> File "app_main.py", line 33, in run_toplevel
> IOError: [Errno 2] No such file or directory: '/compile/pypy/bin/easy_install'
> ----------------------------------------
> Traceback (most recent call last):
> File "app_main.py", line 33, in run_toplevel
> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> line 1647, in <module>
> main()
> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> line 558, in main
> prompt=options.prompt)
> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> line 656, in create_environment
> install_pip(py_executable)
> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> line 415, in install_pip
> filter_stdout=_filter_setup)
> File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> line 624, in call_subprocess
> % (cmd_desc, proc.returncode))
> OSError: Command /compile/pypy/bin/pypy /compile/pypy/bin/easy_install
> /Library/Frameworks/Python.fra...ar.gz failed with error code 1
> 
> 
> The nice thing is that for unittest2 and mock all tests pass with pypy 1.4.
> Not unexpected, but still nice.
> 
> All the best,
> 
> Michael Foord
> 
> -- 
> http://www.voidspace.org.uk

> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

-- 


From fuzzyman at gmail.com  Sat Nov 27 00:22:16 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Fri, 26 Nov 2010 23:22:16 +0000
Subject: [pypy-dev] pypy 1.4 and virtualenv 1.5.1
In-Reply-To: <20101126231940.GJ1009@trillke.net>
References: <AANLkTi=SC_FY-HS-kkzTU4iB_-Mp+qf=rsgGDqf9R5Y8@mail.gmail.com>
	<20101126231940.GJ1009@trillke.net>
Message-ID: <AANLkTikJRT+9gx4mUTsNVzcFw=aB7bRBO-WPtrpDoKcr@mail.gmail.com>

On 26 November 2010 23:19, holger krekel <holger at merlinux.eu> wrote:

> Hi Michael,
>
> On Fri, Nov 26, 2010 at 22:15 +0000, Michael Foord wrote:
> > Congratulations on the new release.
> >
> > Unfortunately it seems that virtualenv 1.5.1 and pypy 1.4 do not play
> well
> > together on Mac OS X. At least the "--distribute" option doesn't work,
> which
> > is needed by tox.
>
> It's not needed, with tox you can configure "distribute=False" in your test
> env
> and this would install setuptools.
>
> > I tried to create an issue on the PyPy issue tracker, but new registrants
> > are unable to create issues:
> >
> > Here is the output on Linux:
> >
> > $ virtualenv -p /compile/pypy-1.4/bin/pypy pypy --distribute
> > --no-site-packages
>
> Does that work for you without --distribute?
>

Yes it does. mvt on #pypy has tracked it down to plat-mac/plist.py not being
included in pypy. This error is somehow being masked when distribute is
installed:


   1. ~/src/distribute-0.6.14 $ mkvirtualenv --distribute --no-site-packages
   --python=/Users/mvantellingen/pypy/bin/pypy pypy-test2
   2. Running virtualenv with interpreter /Users/mvantellingen/pypy/bin/pypy
   3. New pypy-c executable in pypy-test2/bin/pypy
   4. Not overwriting existing pypy-c script pypy-test2/bin/pypy-c (you must
   use pypy-test2/bin/pypy)
   5. Installing distribute...Extracting in
   /var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2
   6. Now working in
   /var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14
   7. Installing Distribute
   8. Traceback (most recent call last):
   9.   File "app_main.py", line 33, in run_toplevel
   10.   File "setup.py", line 37, in <module>
   11.     exec(open(init_path).read(), d)
   12.   File "<string>", line 8, in <module>
   13.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/setuptools/__init__.py",
   line 2, in <module>
   14.     from setuptools.extension import Extension, Library
   15.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/setuptools/extension.py",
   line 2, in <module>
   16.     from setuptools.dist import _get_unpatched
   17.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/setuptools/dist.py",
   line 6, in <module>
   18.     from setuptools.command.install import install
   19.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/setuptools/command/__init__.py",
   line 8, in <module>
   20.     from setuptools.command import install_scripts
   21.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/setuptools/command/install_scripts.py",
   line 3, in <module>
   22.     from pkg_resources import Distribution, PathMetadata,
   ensure_directory
   23.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/pkg_resources.py",
   line 685, in <module>
   24.     class Environment(object):
   25.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/pkg_resources.py",
   line 688, in Environment
   26.     def __init__(self, search_path=None,
   platform=get_supported_platform(), python=PY_MAJOR):
   27.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/pkg_resources.py",
   line 77, in get_supported_platform
   28.     plat = 'macosx-%s-%s' % ('.'.join(_macosx_vers()[:2]),
   m.group(3))
   29.   File
   "/private/var/folders/Dk/DkuwVxyEH4CIjPxKNOHaCE+++TI/-Tmp-/tmpjCepU2/distribute-0.6.14/pkg_resources.py",
   line 192, in _macosx_vers
   30.     import plistlib
   31. ImportError: No module named plistlib
   32. Something went wrong during the installation.
   33. See the error message above.
   34. None
   35. done.
   36.   Complete output from command /Users/mvantellingen/virtualen.../pypy
   /Users/mvantellingen/virtualen...stall
   /Library/Python/2.6/site-packa...ar.gz:
   37.   Traceback (most recent call last):
   38.   File "app_main.py", line 33, in run_toplevel
   39. IOError: [Errno 2] No such file or directory:
   '/Users/mvantellingen/virtualenvs/pypy-test2/bin/easy_install'
   40. ----------------------------------------
   41. Traceback (most recent call last):
   42.   File "app_main.py", line 33, in run_toplevel
   43.   File
   "/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv.py",
   line 1647, in <module>
   44.     main()
   45.   File
   "/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv.py",
   line 558, in main
   46.     prompt=options.prompt)
   47.   File
   "/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv.py",
   line 656, in create_environment
   48.     install_pip(py_executable)
   49.   File
   "/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv.py",
   line 415, in install_pip
   50.     filter_stdout=_filter_setup)
   51.   File
   "/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv.py",
   line 624, in call_subprocess
   52.     % (cmd, proc.returncode))
   53. OSError: Command
   ['/Users/mvantellingen/virtualenvs/pypy-test2/bin/pypy',
   '/Users/mvantellingen/virtualenvs/pypy-test2/bin/easy_install',
   '/Library/Python/2.6/site-packages/virtualenv-1.5.1-py2.6.egg/virtualenv_support/pip-0.8.1.tar.gz']
   failed with error code 1
   54. ~/src/distribute-0.6.14 $


All the best,

Michael Foord


>
> holger
>
> > Running virtualenv with interpreter /compile/pypy-1.4/bin/pypy
> > New pypy-c executable in pypy/bin/pypy
> > Also creating executable in pypy/bin/pypy-c
> > Installing
> >
> distribute....................................................................................................................................................................................done.
> >
> >
> > Here is the failed output on OS X:
> >
> > $ virtualenv -p /compile/pypy-1.4-osx/bin/pypy pypy --distribute
> > --no-site-packages
> > Running virtualenv with interpreter /compile/pypy-1.4-osx/bin/pypy
> > New pypy-c executable in pypy/bin/pypy
> > Also creating executable in pypy/bin/pypy-c
> > Please make sure you remove any previous custom paths from your
> > /Volumes/Second Drive/michael/.pydistutils.cfg file.
> > Installing distribute................................done.
> > Complete output from command /compile/pypy/bin/pypy
> > /compile/pypy/bin/easy_install /Library/Frameworks/Python.fra...ar.gz:
> > Traceback (most recent call last):
> > File "app_main.py", line 33, in run_toplevel
> > IOError: [Errno 2] No such file or directory:
> '/compile/pypy/bin/easy_install'
> > ----------------------------------------
> > Traceback (most recent call last):
> > File "app_main.py", line 33, in run_toplevel
> > File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> > line 1647, in <module>
> > main()
> > File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> > line 558, in main
> > prompt=options.prompt)
> > File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> > line 656, in create_environment
> > install_pip(py_executable)
> > File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> > line 415, in install_pip
> > filter_stdout=_filter_setup)
> > File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/virtualenv.py",
> > line 624, in call_subprocess
> > % (cmd_desc, proc.returncode))
> > OSError: Command /compile/pypy/bin/pypy /compile/pypy/bin/easy_install
> > /Library/Frameworks/Python.fra...ar.gz failed with error code 1
> >
> >
> > The nice thing is that for unittest2 and mock all tests pass with pypy
> 1.4.
> > Not unexpected, but still nice.
> >
> > All the best,
> >
> > Michael Foord
> >
> > --
> > http://www.voidspace.org.uk
>
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
>
> --
>



-- 
http://www.voidspace.org.uk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/26d12411/attachment.html>

From drsalists at gmail.com  Sat Nov 27 01:18:29 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 26 Nov 2010 16:18:29 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CF03ED2.3050801@gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<amauryfa@gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<201011160222.oAG2Mrfv004177@theraft.openend.se>
	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
	<4CF03ED2.3050801@gmail.com>
Message-ID: <AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>

On Fri, Nov 26, 2010 at 3:12 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> On 16/11/10 04:30, Dan Stromberg wrote:
>
>  BTW, it might cause confusion down the road to call something that is
>> basically like cpython's bsddb (Berkeley DB) by the name "dbm" in pypy's
>> library.  In the cpython standard library, "dbm" is an interface to ndbm
>> databases.  These all provide the same dictionary-like interface to Python
>> programs, but have somewhat different API's to C, and pretty different,
>> incompatible on-disk representations.
>>
>
> Hi Dan,
> I played a bit (veeeery quickly) with dbm on both pypy and cpython, and I'm
> not sure I get what you mean when you say that our dbm.py is equivalent to
> cpython's bsddb. E.g., I can create a db on cpython and open it from pypy,
> so it seems that the two modules are compatible.
>
> Moreover, I checked which libraries the links to. On CPython, it links to
> libdb-4.8.so:
>
> viper2 ~ $ ldd /usr/lib/python2.6/lib-dynload/dbm.so
>        linux-gate.so.1 =>  (0x00884000)
>        libdb-4.8.so => /usr/lib/libdb-4.8.so (0x00110000)
>        libpthread.so.0 => /lib/libpthread.so.0 (0x003de000)
>        libc.so.6 => /lib/libc.so.6 (0x003f8000)
>        /lib/ld-linux.so.2 (0x002e0000)
>
> the pypy version first tries to open libdb.so, then libdb-4.5.so. I had to
> manually modify it to open version 4.8 (I agree that we should find a more
> general way to find it), but apart from that what I can see is that it uses
> the same underlying wrapper as CPython.
>
> So, to summarise: could you elaborate a bit more why we should delete
> dbm.py from pypy?
>
> ciao,
> Anto
>


Looks like dbm at the API level: CPython dbm, pypy dbm
Looks like dbm on disk: CPython dbm
Looks like bsddb at the API level: CPython bsddb
Looks like bsddb on disk: CPython bsddb, pypy dbm

Don't let the common prefix fool you - libdb is Berkeley DB, while dbm is
supposed to be ndbm.

http://docs.python.org/library/dbm.html

That is, pypy's dbm.py is perfectly self-consistent (other than a couple of
likely memory leaks), but if you try to open a database from CPython using
pypy's dbm module (or vice-versa), I don't believe it'll work.  EG:

$ /usr/local/cpython-2.7/bin/python
Python 2.7 (r27:82500, Aug  2 2010, 19:15:05)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dbm
>>> d = dbm.open('d', 'n')
>>> d['a'] = 'b'
>>> d.close()
>>>
benchbox-dstromberg:/tmp/dbm-test i686-pc-linux-gnu 30890 - above cmd done
2010 Fri Nov 26 04:14 PM

$ /usr/local/pypy-1.4/bin/pypy
Python 2.5.2 (79529, Nov 25 2010, 20:40:03)
[PyPy 1.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``casuality violations and
flying''
>>>> import dbm
>>>> d = dbm.open('d', 'r')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/pypy-1.4/lib_pypy/dbm.py", line 172, in open
    raise error("Could not open file %s.db" % filename)
error: Could not open file d.db
>>>>

HTH
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/f2cc569e/attachment.html>

From phyo.arkarlwin at gmail.com  Sat Nov 27 01:52:43 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sat, 27 Nov 2010 07:22:43 +0630
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
Message-ID: <AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>

Grats!
i am going to give web2py : http://www.web2py.com a try on pypy

On 11/27/10, Samuele Pedroni <samuele.pedroni at gmail.com> wrote:
> On Fri, Nov 26, 2010 at 7:23 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>> ===============================
>> PyPy 1.4: Ouroboros in practice
>> ===============================
>>
>> We're pleased to announce the 1.4 release of PyPy. This is a major
>> breakthrough
>> in our long journey, as PyPy 1.4 is the first PyPy release that can
>> translate
>> itself faster than CPython.  Starting today, we are using PyPy more for
>> our every-day development.  So may you :) You can download it here:
>>
>>    http://pypy.org/download.html
>>
>> What is PyPy
>> ============
>>
>> PyPy is a very compliant Python interpreter, almost a drop-in replacement
>> for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)
>>
>> Among its new features, this release includes numerous performance
>> improvements
>> (which made fast self-hosting possible), a 64-bit JIT backend, as well
>> as serious stabilization. As of now, we can consider the 32-bit and 64-bit
>> linux versions of PyPy stable enough to run `in production`_.
>>
>> Numerous speed achievements are described on `our blog`_. Normalized speed
>> charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and cpython
>> 2.6`_
>> are available on benchmark website. For the impatient: yes, we got a lot
>> faster!
>>
>
> congratulations!
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From phyo.arkarlwin at gmail.com  Sat Nov 27 02:12:18 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sat, 27 Nov 2010 07:42:18 +0630
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
Message-ID: <AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>

I just tested a simple hello world page in web2py and i dont see speed
improvment yet..

pages comes up in 16-18ms in normal python , and in pypy 22-25


I just run directly via command $pypy web2py.py

am i doing something wrong? need some extra command line options?


On 11/27/10, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
> Grats!
> i am going to give web2py : http://www.web2py.com a try on pypy
>
> On 11/27/10, Samuele Pedroni <samuele.pedroni at gmail.com> wrote:
>> On Fri, Nov 26, 2010 at 7:23 PM, Maciej Fijalkowski <fijall at gmail.com>
>> wrote:
>>> ===============================
>>> PyPy 1.4: Ouroboros in practice
>>> ===============================
>>>
>>> We're pleased to announce the 1.4 release of PyPy. This is a major
>>> breakthrough
>>> in our long journey, as PyPy 1.4 is the first PyPy release that can
>>> translate
>>> itself faster than CPython.  Starting today, we are using PyPy more for
>>> our every-day development.  So may you :) You can download it here:
>>>
>>>    http://pypy.org/download.html
>>>
>>> What is PyPy
>>> ============
>>>
>>> PyPy is a very compliant Python interpreter, almost a drop-in
>>> replacement
>>> for CPython. It's fast (`pypy 1.4 and cpython 2.6`_ comparison)
>>>
>>> Among its new features, this release includes numerous performance
>>> improvements
>>> (which made fast self-hosting possible), a 64-bit JIT backend, as well
>>> as serious stabilization. As of now, we can consider the 32-bit and
>>> 64-bit
>>> linux versions of PyPy stable enough to run `in production`_.
>>>
>>> Numerous speed achievements are described on `our blog`_. Normalized
>>> speed
>>> charts comparing `pypy 1.4 and pypy 1.3`_ as well as `pypy 1.4 and
>>> cpython
>>> 2.6`_
>>> are available on benchmark website. For the impatient: yes, we got a lot
>>> faster!
>>>
>>
>> congratulations!
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>


From santagada at gmail.com  Sat Nov 27 02:56:09 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 26 Nov 2010 23:56:09 -0200
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
Message-ID: <AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>

On Fri, Nov 26, 2010 at 11:12 PM, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
> I just tested a simple hello world page in web2py and i dont see speed
> improvment yet..
>
> pages comes up in 16-18ms in normal python , and in pypy 22-25
>
>
> I just run directly via command $pypy web2py.py
>
> am i doing something wrong? need some extra command line options?

The pypy jit is always on, what you could do is maybe warm up the
page, like making 100 requests and then measuring the next 10 requests
would make a more interesting benchmark.


-- 
Leonardo Santagada


From phyo.arkarlwin at gmail.com  Sat Nov 27 03:09:15 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sat, 27 Nov 2010 08:39:15 +0630
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
Message-ID: <AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>

libmagic python fails to work on pypy (python-magic)

it uses ctypes and easy-install try to download and instaill it , but it fails.

how to enable ctypes on pypy?


On 11/27/10, Leonardo Santagada <santagada at gmail.com> wrote:
> On Fri, Nov 26, 2010 at 11:12 PM, Phyo Arkar <phyo.arkarlwin at gmail.com>
> wrote:
>> I just tested a simple hello world page in web2py and i dont see speed
>> improvment yet..
>>
>> pages comes up in 16-18ms in normal python , and in pypy 22-25
>>
>>
>> I just run directly via command $pypy web2py.py
>>
>> am i doing something wrong? need some extra command line options?
>
> The pypy jit is always on, what you could do is maybe warm up the
> page, like making 100 requests and then measuring the next 10 requests
> would make a more interesting benchmark.
>
>
> --
> Leonardo Santagada
>


From drsalists at gmail.com  Sat Nov 27 03:51:57 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 26 Nov 2010 18:51:57 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<amauryfa@gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<201011160222.oAG2Mrfv004177@theraft.openend.se>
	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
	<4CF03ED2.3050801@gmail.com>
	<AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>
Message-ID: <AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>

On Fri, Nov 26, 2010 at 4:18 PM, Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Fri, Nov 26, 2010 at 3:12 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>
>> On 16/11/10 04:30, Dan Stromberg wrote:
>>
>>  BTW, it might cause confusion down the road to call something that is
>>> basically like cpython's bsddb (Berkeley DB) by the name "dbm" in pypy's
>>> library.  In the cpython standard library, "dbm" is an interface to ndbm
>>> databases.  These all provide the same dictionary-like interface to
>>> Python
>>> programs, but have somewhat different API's to C, and pretty different,
>>> incompatible on-disk representations.
>>>
>>
>> Hi Dan,
>> I played a bit (veeeery quickly) with dbm on both pypy and cpython, and
>> I'm not sure I get what you mean when you say that our dbm.py is equivalent
>> to cpython's bsddb. E.g., I can create a db on cpython and open it from
>> pypy, so it seems that the two modules are compatible.
>>
>> Moreover, I checked which libraries the links to. On CPython, it links to
>> libdb-4.8.so:
>>
>> viper2 ~ $ ldd /usr/lib/python2.6/lib-dynload/dbm.so
>>        linux-gate.so.1 =>  (0x00884000)
>>        libdb-4.8.so => /usr/lib/libdb-4.8.so (0x00110000)
>>        libpthread.so.0 => /lib/libpthread.so.0 (0x003de000)
>>        libc.so.6 => /lib/libc.so.6 (0x003f8000)
>>        /lib/ld-linux.so.2 (0x002e0000)
>>
>> the pypy version first tries to open libdb.so, then libdb-4.5.so. I had
>> to manually modify it to open version 4.8 (I agree that we should find a
>> more general way to find it), but apart from that what I can see is that it
>> uses the same underlying wrapper as CPython.
>>
>> So, to summarise: could you elaborate a bit more why we should delete
>> dbm.py from pypy?
>>
>> ciao,
>> Anto
>>
>
>
> Looks like dbm at the API level: CPython dbm, pypy dbm
> Looks like dbm on disk: CPython dbm
> Looks like bsddb at the API level: CPython bsddb
> Looks like bsddb on disk: CPython bsddb, pypy dbm
>
> Don't let the common prefix fool you - libdb is Berkeley DB, while dbm is
> supposed to be ndbm.
>
> http://docs.python.org/library/dbm.html
>
> That is, pypy's dbm.py is perfectly self-consistent (other than a couple of
> likely memory leaks), but if you try to open a database from CPython using
> pypy's dbm module (or vice-versa), I don't believe it'll work.  EG:
>
> $ /usr/local/cpython-2.7/bin/python
> Python 2.7 (r27:82500, Aug  2 2010, 19:15:05)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import dbm
> >>> d = dbm.open('d', 'n')
> >>> d['a'] = 'b'
> >>> d.close()
> >>>
> benchbox-dstromberg:/tmp/dbm-test i686-pc-linux-gnu 30890 - above cmd done
> 2010 Fri Nov 26 04:14 PM
>
> $ /usr/local/pypy-1.4/bin/pypy
> Python 2.5.2 (79529, Nov 25 2010, 20:40:03)
> [PyPy 1.4.0] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> And now for something completely different: ``casuality violations and
> flying''
> >>>> import dbm
> >>>> d = dbm.open('d', 'r')
> Traceback (most recent call last):
>   File "<console>", line 1, in <module>
>   File "/usr/local/pypy-1.4/lib_pypy/dbm.py", line 172, in open
>     raise error("Could not open file %s.db" % filename)
> error: Could not open file d.db
> >>>>
>
> HTH
>
> Interesting.  My CPython 2.7 build has:

$ ldd dbm.so
        linux-gate.so.1 =>  (0x009ed000)
        libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0x00ed5000)
        libgdbm_compat.so.3 => /usr/lib/libgdbm_compat.so.3 (0x00269000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00df3000)
        libc.so.6 => /lib/libc.so.6 (0x00425000)
        /lib/ld-linux.so.2 (0x00b7b000)
benchbox-dstromberg:/usr/local/cpython-2.7/lib/python2.7/lib-dynload
i686-pc-linux-gnu 30430 - above cmd done 2010 Fri Nov 26 06:48 PM

...but http://docs.python.org/library/dbm.html plainly says it should be
ndbm.

So which is wrong?  The doc, or the module that's picking gdbm or Berkeley
DB, as it sees fit?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/a352b5be/attachment.html>

From drsalists at gmail.com  Sat Nov 27 04:07:49 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 26 Nov 2010 19:07:49 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<amauryfa@gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<201011160222.oAG2Mrfv004177@theraft.openend.se>
	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
	<4CF03ED2.3050801@gmail.com>
	<AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>
	<AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>
Message-ID: <AANLkTincyJPhS_wm92eD3fTMDJtcqJ0ruSEytm6yKZ2W@mail.gmail.com>

On Fri, Nov 26, 2010 at 6:51 PM, Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Fri, Nov 26, 2010 at 4:18 PM, Dan Stromberg <drsalists at gmail.com>wrote:
>
>>
>> On Fri, Nov 26, 2010 at 3:12 PM, Antonio Cuni <anto.cuni at gmail.com>wrote:
>>
>>> On 16/11/10 04:30, Dan Stromberg wrote:
>>>
>>>  BTW, it might cause confusion down the road to call something that is
>>>> basically like cpython's bsddb (Berkeley DB) by the name "dbm" in pypy's
>>>> library.  In the cpython standard library, "dbm" is an interface to ndbm
>>>> databases.  These all provide the same dictionary-like interface to
>>>> Python
>>>> programs, but have somewhat different API's to C, and pretty different,
>>>> incompatible on-disk representations.
>>>>
>>>
>>> Hi Dan,
>>> I played a bit (veeeery quickly) with dbm on both pypy and cpython, and
>>> I'm not sure I get what you mean when you say that our dbm.py is equivalent
>>> to cpython's bsddb. E.g., I can create a db on cpython and open it from
>>> pypy, so it seems that the two modules are compatible.
>>>
>>> Moreover, I checked which libraries the links to. On CPython, it links to
>>> libdb-4.8.so:
>>>
>>> viper2 ~ $ ldd /usr/lib/python2.6/lib-dynload/dbm.so
>>>        linux-gate.so.1 =>  (0x00884000)
>>>        libdb-4.8.so => /usr/lib/libdb-4.8.so (0x00110000)
>>>        libpthread.so.0 => /lib/libpthread.so.0 (0x003de000)
>>>        libc.so.6 => /lib/libc.so.6 (0x003f8000)
>>>        /lib/ld-linux.so.2 (0x002e0000)
>>>
>>> the pypy version first tries to open libdb.so, then libdb-4.5.so. I had
>>> to manually modify it to open version 4.8 (I agree that we should find a
>>> more general way to find it), but apart from that what I can see is that it
>>> uses the same underlying wrapper as CPython.
>>>
>>> So, to summarise: could you elaborate a bit more why we should delete
>>> dbm.py from pypy?
>>>
>>> ciao,
>>> Anto
>>>
>>
>>
>> Looks like dbm at the API level: CPython dbm, pypy dbm
>> Looks like dbm on disk: CPython dbm
>> Looks like bsddb at the API level: CPython bsddb
>> Looks like bsddb on disk: CPython bsddb, pypy dbm
>>
>> Don't let the common prefix fool you - libdb is Berkeley DB, while dbm is
>> supposed to be ndbm.
>>
>> http://docs.python.org/library/dbm.html
>>
>> That is, pypy's dbm.py is perfectly self-consistent (other than a couple
>> of likely memory leaks), but if you try to open a database from CPython
>> using pypy's dbm module (or vice-versa), I don't believe it'll work.  EG:
>>
>> $ /usr/local/cpython-2.7/bin/python
>> Python 2.7 (r27:82500, Aug  2 2010, 19:15:05)
>> [GCC 4.4.3] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> import dbm
>> >>> d = dbm.open('d', 'n')
>> >>> d['a'] = 'b'
>> >>> d.close()
>> >>>
>> benchbox-dstromberg:/tmp/dbm-test i686-pc-linux-gnu 30890 - above cmd done
>> 2010 Fri Nov 26 04:14 PM
>>
>> $ /usr/local/pypy-1.4/bin/pypy
>> Python 2.5.2 (79529, Nov 25 2010, 20:40:03)
>> [PyPy 1.4.0] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> And now for something completely different: ``casuality violations and
>> flying''
>> >>>> import dbm
>> >>>> d = dbm.open('d', 'r')
>> Traceback (most recent call last):
>>   File "<console>", line 1, in <module>
>>   File "/usr/local/pypy-1.4/lib_pypy/dbm.py", line 172, in open
>>     raise error("Could not open file %s.db" % filename)
>> error: Could not open file d.db
>> >>>>
>>
>> HTH
>>
>> Interesting.  My CPython 2.7 build has:
>
> $ ldd dbm.so
>         linux-gate.so.1 =>  (0x009ed000)
>         libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0x00ed5000)
>         libgdbm_compat.so.3 => /usr/lib/libgdbm_compat.so.3 (0x00269000)
>         libpthread.so.0 => /lib/libpthread.so.0 (0x00df3000)
>         libc.so.6 => /lib/libc.so.6 (0x00425000)
>         /lib/ld-linux.so.2 (0x00b7b000)
> benchbox-dstromberg:/usr/local/cpython-2.7/lib/python2.7/lib-dynload
> i686-pc-linux-gnu 30430 - above cmd done 2010 Fri Nov 26 06:48 PM
>
> ...but http://docs.python.org/library/dbm.html plainly says it should be
> ndbm.
>
> So which is wrong?  The doc, or the module that's picking gdbm or Berkeley
> DB, as it sees fit?
>
>
But then there's this at the URL above:

This module can be used with the ?classic? ndbm interface, the BSD DB
compatibility interface, or the GNU GDBM compatibility interface. On Unix,
the *configure* script will attempt to locate the appropriate header file to
simplify building this module.

I suppose that means that if it can't find ndbm (which at one time was hard
due to licensing, but last I heard it'd become readily available), it's free
to pretend it has ndbm using something else.

I'd call that puzzlingly worded - it's not the interface that's changing,
but the backend implementation.  But perhaps dbm.py is free to use Berkeley
DB if it prefers to pretend it can never find ndbm.  And perhaps I shouldn't
have skimmed that page so quickly ^_^

CPython 2.7's configure script has:

  --with-dbmliborder=db1:db2:...
                          order to check db backends for dbm. Valid value is
a
                          colon separated string with the backend names
                          `ndbm', `gdbm' and `bdb'.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/c4a1d226/attachment.html>

From ademan555 at gmail.com  Sat Nov 27 05:47:21 2010
From: ademan555 at gmail.com (Dan Roberts)
Date: Fri, 26 Nov 2010 20:47:21 -0800
Subject: [pypy-dev] test_all.py -A behavior
In-Reply-To: <AANLkTi=nGojBHmdBM+4+FNNxiGTC1yZoNtd5PvcUj-a4@mail.gmail.com>
References: <AANLkTinrKBxRBFSa68T2EVrZiv4F8K1YuCLyZxEKGUU1@mail.gmail.com>
	<AANLkTi=dOge6QeVYAF8ZseH0vw3T5Wcm2v1GcyZT_P2a@mail.gmail.com>
	<AANLkTi=O6v_kA9k2t08m_-6TXbVgbF=pXE3GPKSHF6wr@mail.gmail.com>
	<AANLkTi=nGojBHmdBM+4+FNNxiGTC1yZoNtd5PvcUj-a4@mail.gmail.com>
Message-ID: <AANLkTimOFEAFSoZsw3U=Va3KnR3P4SBk96F2K6AfJMT1@mail.gmail.com>

I'm having trouble running the applevel tests on a translated pypy-c
(without jit).  I have this problem with recent trunk and my psycopg2
compatibility branch. Translating both with:

python translate.py targetpypystandalone.py --withmod-cpyext

and running tests with:

translator/goal/pypy-c test_all.py -A

both trunk and my branch have multiple errors on
module/bz2/test/test_bz2_compdecomp.py tests, one failure in
module/posix/test/test_posix2.py and both hang on the 8th test on
module/signal/test/test_signal.py .  This wouldn't be too weird if all three
of those test suites run successfully via:

translator/goal/pypy-c test_all.py -A $SUITE

on both my branch and trunk.  I'm on Ubuntu 10.04 32 bit.  I tried again
with trunk just now with the same behavior.  Is this a test_all.py bug?  Can
anyone with a pypy-c confirm this behavior?

Cheers,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101126/f8c43e20/attachment.html>

From anto.cuni at gmail.com  Sat Nov 27 07:17:59 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 27 Nov 2010 07:17:59 +0100
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
Message-ID: <4CF0A297.9090202@gmail.com>

On 27/11/10 03:09, Phyo Arkar wrote:
> libmagic python fails to work on pypy (python-magic)
>
> it uses ctypes and easy-install try to download and instaill it , but it fails.
>
> how to enable ctypes on pypy?

Hi Phyo,
ctypes *is* enabled on pypy by default.

If python-magic does not work, it can either:

1) be a pypy bug: please report it as an issue (possibly with a simple failing 
test and the full traceack)

2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot 
really be supported by pypy due to the internal differences with CPython

ciao,
Anto


From phyo.arkarlwin at gmail.com  Sun Nov 28 09:57:26 2010
From: phyo.arkarlwin at gmail.com (Phyo Arkar)
Date: Sun, 28 Nov 2010 15:27:26 +0630
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <4CF0A297.9090202@gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
	<4CF0A297.9090202@gmail.com>
Message-ID: <AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>

i got python-magic working , after i installed without easy_install
(easy_install fail because it tried to install ctypes).

Now what is not working is python-lxml , which is very important for my project.

here are the errors:

Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2
Building lxml version 2.3.beta1.
NOTE: Trying to build without Cython, pre-generated
'src/lxml/lxml.etree.c' needs to be available.
Using build configuration of libxslt 1.1.26
Building against libxml2/libxslt in the following directory: /usr/lib
src/lxml/lxml.etree.c:75: error: conflicting types for ?Py_buffer?
/home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of
?Py_buffer? was here


Had Anyone got lxml working in pypy successfully?

On 11/27/10, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On 27/11/10 03:09, Phyo Arkar wrote:
>> libmagic python fails to work on pypy (python-magic)
>>
>> it uses ctypes and easy-install try to download and instaill it , but it
>> fails.
>>
>> how to enable ctypes on pypy?
>
> Hi Phyo,
> ctypes *is* enabled on pypy by default.
>
> If python-magic does not work, it can either:
>
> 1) be a pypy bug: please report it as an issue (possibly with a simple
> failing
> test and the full traceack)
>
> 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot
> really be supported by pypy due to the internal differences with CPython
>
> ciao,
> Anto
>


From fijall at gmail.com  Sun Nov 28 10:48:17 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 28 Nov 2010 11:48:17 +0200
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
	<4CF0A297.9090202@gmail.com>
	<AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>
Message-ID: <AANLkTimuhqCt5WkgcrqovR1gHN09LrUz+ZAsNc8czUjP@mail.gmail.com>

Hey.

On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar <phyo.arkarlwin at gmail.com> wrote:
> i got python-magic working , after i installed without easy_install
> (easy_install fail because it tried to install ctypes).

great

>
> Now what is not working is python-lxml , which is very important for my project.

lxml won't work out of the box. if you think it's important enough,
you can try to port cython to generate something saner (right now what
it generates won't work on cpyext).

Cheers,
fijal

>
> here are the errors:
>
> Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir
> /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2
> Building lxml version 2.3.beta1.
> NOTE: Trying to build without Cython, pre-generated
> 'src/lxml/lxml.etree.c' needs to be available.
> Using build configuration of libxslt 1.1.26
> Building against libxml2/libxslt in the following directory: /usr/lib
> src/lxml/lxml.etree.c:75: error: conflicting types for ?Py_buffer?
> /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of
> ?Py_buffer? was here
>
>
> Had Anyone got lxml working in pypy successfully?
>
> On 11/27/10, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> On 27/11/10 03:09, Phyo Arkar wrote:
>>> libmagic python fails to work on pypy (python-magic)
>>>
>>> it uses ctypes and easy-install try to download and instaill it , but it
>>> fails.
>>>
>>> how to enable ctypes on pypy?
>>
>> Hi Phyo,
>> ctypes *is* enabled on pypy by default.
>>
>> If python-magic does not work, it can either:
>>
>> 1) be a pypy bug: please report it as an issue (possibly with a simple
>> failing
>> test and the full traceack)
>>
>> 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot
>> really be supported by pypy due to the internal differences with CPython
>>
>> ciao,
>> Anto
>>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From renesd at gmail.com  Sun Nov 28 10:58:33 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Sun, 28 Nov 2010 09:58:33 +0000
Subject: [pypy-dev] which xml libraries? was (Re:  PyPy 1.4 released)
Message-ID: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>

Hi,

what xml libraries are people using with pypy?  What is working well?

cu,


On Sun, Nov 28, 2010 at 9:48 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> Hey.
>
> On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar <phyo.arkarlwin at gmail.com>
> wrote:
> > i got python-magic working , after i installed without easy_install
> > (easy_install fail because it tried to install ctypes).
>
> great
>
> >
> > Now what is not working is python-lxml , which is very important for my
> project.
>
> lxml won't work out of the box. if you think it's important enough,
> you can try to port cython to generate something saner (right now what
> it generates won't work on cpyext).
>
> Cheers,
> fijal
>
> >
> > here are the errors:
> >
> > Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir
> > /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2
> > Building lxml version 2.3.beta1.
> > NOTE: Trying to build without Cython, pre-generated
> > 'src/lxml/lxml.etree.c' needs to be available.
> > Using build configuration of libxslt 1.1.26
> > Building against libxml2/libxslt in the following directory: /usr/lib
> > src/lxml/lxml.etree.c:75: error: conflicting types for ?Py_buffer?
> > /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of
> > ?Py_buffer? was here
> >
> >
> > Had Anyone got lxml working in pypy successfully?
> >
> > On 11/27/10, Antonio Cuni <anto.cuni at gmail.com> wrote:
> >> On 27/11/10 03:09, Phyo Arkar wrote:
> >>> libmagic python fails to work on pypy (python-magic)
> >>>
> >>> it uses ctypes and easy-install try to download and instaill it , but
> it
> >>> fails.
> >>>
> >>> how to enable ctypes on pypy?
> >>
> >> Hi Phyo,
> >> ctypes *is* enabled on pypy by default.
> >>
> >> If python-magic does not work, it can either:
> >>
> >> 1) be a pypy bug: please report it as an issue (possibly with a simple
> >> failing
> >> test and the full traceack)
> >>
> >> 2) a python-magic issue, e.g. if it plays dirty ctypes trick that cannot
> >> really be supported by pypy due to the internal differences with CPython
> >>
> >> ciao,
> >> Anto
> >>
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
> >
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101128/61b91832/attachment.html>

From fijall at gmail.com  Sun Nov 28 11:09:25 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 28 Nov 2010 12:09:25 +0200
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
Message-ID: <AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>

On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield <renesd at gmail.com> wrote:
> Hi,
>
> what xml libraries are people using with pypy?? What is working well?
>
> cu,

PyExpat works, although it's slow (ctypes-based implementation). I
know genshi has some troubles with it, someone is debugging now.
Besides I don't think there are any working (unless someone wrote a
pure-python one)

Cheers,
fijal

>
>
> On Sun, Nov 28, 2010 at 9:48 AM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> Hey.
>>
>> On Sun, Nov 28, 2010 at 10:57 AM, Phyo Arkar <phyo.arkarlwin at gmail.com>
>> wrote:
>> > i got python-magic working , after i installed without easy_install
>> > (easy_install fail because it tried to install ctypes).
>>
>> great
>>
>> >
>> > Now what is not working is python-lxml , which is very important for my
>> > project.
>>
>> lxml won't work out of the box. if you think it's important enough,
>> you can try to port cython to generate something saner (right now what
>> it generates won't work on cpyext).
>>
>> Cheers,
>> fijal
>>
>> >
>> > here are the errors:
>> >
>> > Running lxml-2.3beta1/setup.py -q bdist_egg --dist-dir
>> > /tmp/easy_install-Gg3GRA/lxml-2.3beta1/egg-dist-tmp-bwUkM2
>> > Building lxml version 2.3.beta1.
>> > NOTE: Trying to build without Cython, pre-generated
>> > 'src/lxml/lxml.etree.c' needs to be available.
>> > Using build configuration of libxslt 1.1.26
>> > Building against libxml2/libxslt in the following directory: /usr/lib
>> > src/lxml/lxml.etree.c:75: error: conflicting types for ?Py_buffer?
>> > /home/v3ss/pypy-1.4/include/object.h:19: note: previous declaration of
>> > ?Py_buffer? was here
>> >
>> >
>> > Had Anyone got lxml working in pypy successfully?
>> >
>> > On 11/27/10, Antonio Cuni <anto.cuni at gmail.com> wrote:
>> >> On 27/11/10 03:09, Phyo Arkar wrote:
>> >>> libmagic python fails to work on pypy (python-magic)
>> >>>
>> >>> it uses ctypes and easy-install try to download and instaill it , but
>> >>> it
>> >>> fails.
>> >>>
>> >>> how to enable ctypes on pypy?
>> >>
>> >> Hi Phyo,
>> >> ctypes *is* enabled on pypy by default.
>> >>
>> >> If python-magic does not work, it can either:
>> >>
>> >> 1) be a pypy bug: please report it as an issue (possibly with a simple
>> >> failing
>> >> test and the full traceack)
>> >>
>> >> 2) a python-magic issue, e.g. if it plays dirty ctypes trick that
>> >> cannot
>> >> really be supported by pypy due to the internal differences with
>> >> CPython
>> >>
>> >> ciao,
>> >> Anto
>> >>
>> > _______________________________________________
>> > pypy-dev at codespeak.net
>> > http://codespeak.net/mailman/listinfo/pypy-dev
>> >
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From amauryfa at gmail.com  Sun Nov 28 11:44:40 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sun, 28 Nov 2010 11:44:40 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
Message-ID: <AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>

Hi

2010/11/28 Maciej Fijalkowski <fijall at gmail.com>

> On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield <renesd at gmail.com> wrote:
> > what xml libraries are people using with pypy?  What is working well?
>
> PyExpat works, although it's slow (ctypes-based implementation). I
> know genshi has some troubles with it, someone is debugging now.
> Besides I don't think there are any working (unless someone wrote a
> pure-python one)
>

PyExpat is now a built-in module, implemented in RPython,
and should have reasonable performance.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101128/5074733d/attachment.html>

From anto.cuni at gmail.com  Sun Nov 28 12:10:19 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sun, 28 Nov 2010 12:10:19 +0100
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <AANLkTimuhqCt5WkgcrqovR1gHN09LrUz+ZAsNc8czUjP@mail.gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
	<4CF0A297.9090202@gmail.com>
	<AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>
	<AANLkTimuhqCt5WkgcrqovR1gHN09LrUz+ZAsNc8czUjP@mail.gmail.com>
Message-ID: <4CF2389B.90801@gmail.com>

On 28/11/10 10:48, Maciej Fijalkowski wrote:
>> > i got python-magic working , after i installed without easy_install
>> > (easy_install fail because it tried to install ctypes).

> great


well, that's still a bit strange. Why does easy_install try to install ctypes,
given that it's in the stdlib?


From renesd at gmail.com  Sun Nov 28 13:19:30 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Sun, 28 Nov 2010 12:19:30 +0000
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <4CF2389B.90801@gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
	<4CF0A297.9090202@gmail.com>
	<AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>
	<AANLkTimuhqCt5WkgcrqovR1gHN09LrUz+ZAsNc8czUjP@mail.gmail.com>
	<4CF2389B.90801@gmail.com>
Message-ID: <AANLkTik2BOqL0rRi7umav+YKq01Ob7iK73_DkQX0U74G@mail.gmail.com>

Maybe it's doing a version comparison?

On Sun, Nov 28, 2010 at 11:10 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> On 28/11/10 10:48, Maciej Fijalkowski wrote:
> >> > i got python-magic working , after i installed without easy_install
> >> > (easy_install fail because it tried to install ctypes).
>
> > great
>
>
> well, that's still a bit strange. Why does easy_install try to install
> ctypes,
> given that it's in the stdlib?
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101128/a1b0f80b/attachment.html>

From anto.cuni at gmail.com  Mon Nov 29 10:00:09 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 29 Nov 2010 10:00:09 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTincyJPhS_wm92eD3fTMDJtcqJ0ruSEytm6yKZ2W@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<amauryfa@gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>	<201011160222.oAG2Mrfv004177@theraft.openend.se>	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>	<4CF03ED2.3050801@gmail.com>	<AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>	<AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>
	<AANLkTincyJPhS_wm92eD3fTMDJtcqJ0ruSEytm6yKZ2W@mail.gmail.com>
Message-ID: <4CF36B99.9080104@gmail.com>

On 27/11/10 04:07, Dan Stromberg wrote:

> This module can be used with the ?classic? ndbm interface, the BSD DB
> compatibility interface, or the GNU GDBM compatibility interface. On Unix, the
> *configure* script will attempt to locate the appropriate header file to
> simplify building this module.
> 
> I suppose that means that if it can't find ndbm (which at one time was hard
> due to licensing, but last I heard it'd become readily available), it's free
> to pretend it has ndbm using something else.
> 
> I'd call that puzzlingly worded - it's not the interface that's changing, but
> the backend implementation.  But perhaps dbm.py is free to use Berkeley DB if
> it prefers to pretend it can never find ndbm.  And perhaps I shouldn't have
> skimmed that page so quickly ^_^
> 
> CPython 2.7's configure script has:
> 
>   --with-dbmliborder=db1:db2:...
>                           order to check db backends for dbm. Valid value is a
>                           colon separated string with the backend names
>                           `ndbm', `gdbm' and `bdb'.
> 
> 

so, having a dbm.py which links to libdb is fine, and it's also what you get
with cpython on ubuntu.  There is still the issue of how to find the correct
library name, as it seems it can vary (it was db-4.5 when the module was
written, it's db-4.8 nowadays), but this is a bit orthogonal to what we are
discussing here.

To summarize, I think we should keep the current dbm.py to link against libdb,
and integrate your gdbm.py to link against libgdbm.  But before merging it to
trunk, I'd like to solve the issue of code duplication between the two modules.

ciao,
Anto


From arigo at tunes.org  Mon Nov 29 11:32:59 2010
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 29 Nov 2010 11:32:59 +0100
Subject: [pypy-dev] PyPy-JIT memory usage compared to CPython?
In-Reply-To: <160969.12027.qm@web111315.mail.gq1.yahoo.com>
References: <160969.12027.qm@web111315.mail.gq1.yahoo.com>
Message-ID: <AANLkTi=HSn6x127NyDV5WdQD=-k4eh0Gmoo5ht6vXeVi@mail.gmail.com>

Hi,

On Sat, Nov 27, 2010 at 12:03 AM, Andy <angelflow at yahoo.com> wrote:
> I'm particularly interested in the memory usage of PyPy-JIT vs. CPython when running Django. In general would PyPy-JIT increase or decrease memory usage?

It depends on the kind of application.  PyPy with a JIT may get to
1.5x or 2x more than CPython, but it may also be smaller than CPython
if your program contains mainly instances and you didn't feel like
carefully putting __slots__ everywhere.


A bient?t,

Armin.


From arigo at tunes.org  Mon Nov 29 11:38:01 2010
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 29 Nov 2010 11:38:01 +0100
Subject: [pypy-dev] test_all.py -A behavior
In-Reply-To: <AANLkTimOFEAFSoZsw3U=Va3KnR3P4SBk96F2K6AfJMT1@mail.gmail.com>
References: <AANLkTinrKBxRBFSa68T2EVrZiv4F8K1YuCLyZxEKGUU1@mail.gmail.com>
	<AANLkTi=dOge6QeVYAF8ZseH0vw3T5Wcm2v1GcyZT_P2a@mail.gmail.com>
	<AANLkTi=O6v_kA9k2t08m_-6TXbVgbF=pXE3GPKSHF6wr@mail.gmail.com>
	<AANLkTi=nGojBHmdBM+4+FNNxiGTC1yZoNtd5PvcUj-a4@mail.gmail.com>
	<AANLkTimOFEAFSoZsw3U=Va3KnR3P4SBk96F2K6AfJMT1@mail.gmail.com>
Message-ID: <AANLkTi=fqhwLf7O_KSRboP9X_VSZJUr+Dkfwe0VJBTd6@mail.gmail.com>

Hi Dan,

On Sat, Nov 27, 2010 at 5:47 AM, Dan Roberts <ademan555 at gmail.com> wrote:
> I'm having trouble running the applevel tests on a translated pypy-c
> (without jit).? I have this problem with recent trunk and my psycopg2
> compatibility branch. Translating both with:
>
> python translate.py targetpypystandalone.py --withmod-cpyext
>
> and running tests with:
>
> translator/goal/pypy-c test_all.py -A

We run these tests nightly, and they seem to pass.  You'll have to
come to IRC to discuss this, or else post much more details about your
problem -- e.g. does it work in a plain trunk without your
compatibility branch?


A bient?t,

Armin.


From wlavrijsen at lbl.gov  Mon Nov 29 14:04:22 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 29 Nov 2010 05:04:22 -0800 (PST)
Subject: [pypy-dev] double trouble with JIT
Message-ID: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>

Hi,

with the Reflex branch, and the fast path enabled, I sometimes run into
pbs with doubles in libffi. Sometimes the results are wrong, sometimes
there's a crash, and sometimes there's:

self = <C object ffi_cif (opaque) at 0xbfb8848>

     def _ctypes_storage_was_allocated(self):
         addr = ctypes.cast(self._storage, ctypes.c_void_p).value
         if addr in ALLOCATED:
>           raise Exception("internal ll2ctypes error - "
                             "double conversion from lltype to ctypes?")
E           Exception: internal ll2ctypes error - double conversion from lltype to ctypes?

Any ideas?

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From cfbolz at gmx.de  Mon Nov 29 14:12:04 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 29 Nov 2010 14:12:04 +0100
Subject: [pypy-dev] double trouble with JIT
In-Reply-To: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
Message-ID: <4CF3A6A4.7080607@gmx.de>

On 11/29/2010 02:04 PM, wlavrijsen at lbl.gov wrote:
> Hi,
>
> with the Reflex branch, and the fast path enabled, I sometimes run into
> pbs with doubles in libffi. Sometimes the results are wrong, sometimes
> there's a crash, and sometimes there's:
>
> self =<C object ffi_cif (opaque) at 0xbfb8848>
>
>       def _ctypes_storage_was_allocated(self):
>           addr = ctypes.cast(self._storage, ctypes.c_void_p).value
>           if addr in ALLOCATED:
>>            raise Exception("internal ll2ctypes error - "
>                               "double conversion from lltype to ctypes?")
> E           Exception: internal ll2ctypes error - double conversion from lltype to ctypes?

Could you check in a test that shows the behaviour so that I can 
reproduce that? It is definitely possible that there is a bug in one of 
the layers that has nothing to do with cppyy.

Cheers,

Carl Friedrich


From wlavrijsen at lbl.gov  Mon Nov 29 14:17:12 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 29 Nov 2010 05:17:12 -0800 (PST)
Subject: [pypy-dev] double trouble with JIT
In-Reply-To: <4CF3A6A4.7080607@gmx.de>
References: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
	<4CF3A6A4.7080607@gmx.de>
Message-ID: <alpine.LNX.2.00.1011290514450.2101@pc3-wlav.lbl.gov>

Hi Carl,

> Could you check in a test that shows the behaviour so that I can
> reproduce that? It is definitely possible that there is a bug in one of
> the layers that has nothing to do with cppyy.

simply run:

   $ cd pypy/module/cppyy/test
   $ pypy-c-jit ../../../test_all.py -x test_cppyy.py

You may have to run it a few times to get the exception, rather than any of
the other errors.

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From stefan_ml at behnel.de  Mon Nov 29 14:40:49 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 29 Nov 2010 14:40:49 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
Message-ID: <id0ah2$2pr$1@dough.gmane.org>

Amaury Forgeot d'Arc, 28.11.2010 11:44:
> 2010/11/28 Maciej Fijalkowski
>
>> On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield wrote:
>>> what xml libraries are people using with pypy?  What is working well?
>>
>> PyExpat works, although it's slow (ctypes-based implementation). I
>> know genshi has some troubles with it, someone is debugging now.
>> Besides I don't think there are any working (unless someone wrote a
>> pure-python one)
>
> PyExpat is now a built-in module, implemented in RPython,
> and should have reasonable performance.

Hmm, reasonable?

$ ./bin/pypy -m timeit -s 'import xml.etree.ElementTree as ET' \
      'ET.parse("ot.xml")'
10 loops, best of 3: 1.27 sec per loop

$ python2.7 -m timeit -s 'import xml.etree.ElementTree as ET' \
      'ET.parse("ot.xml")'
10 loops, best of 3: 486 msec per loop

$ python2.7 -m timeit -s 'import xml.etree.cElementTree as ET' \
      'ET.parse("ot.xml")'
10 loops, best of 3: 33.7 msec per loop


Stefan



From cfbolz at gmx.de  Mon Nov 29 14:50:59 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 29 Nov 2010 14:50:59 +0100
Subject: [pypy-dev] double trouble with JIT
In-Reply-To: <alpine.LNX.2.00.1011290514450.2101@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
	<4CF3A6A4.7080607@gmx.de>
	<alpine.LNX.2.00.1011290514450.2101@pc3-wlav.lbl.gov>
Message-ID: <4CF3AFC3.1040703@gmx.de>

Hi Wim,

On 11/29/2010 02:17 PM, wlavrijsen at lbl.gov wrote:
>> Could you check in a test that shows the behaviour so that I can
>> reproduce that? It is definitely possible that there is a bug in one of
>> the layers that has nothing to do with cppyy.
>
> simply run:
>
> $ cd pypy/module/cppyy/test
> $ pypy-c-jit ../../../test_all.py -x test_cppyy.py
>
> You may have to run it a few times to get the exception, rather than any of
> the other errors.

Given that we are not running PyPy's test suite on PyPy nightly 
currently, I think it would be safer to keep running the tests on top of 
CPython. Running the tests needs ctypes, and I don't quite trust PyPy's 
implementation of that yet (there are even known bugs that have not been 
fixed on trunk but only on a branch).

You should still be able to use pypy-c-jit for translation, because for 
translation (and after translation) you won't need ctypes that much.

Cheers,

Carl Friedrich


From renesd at gmail.com  Mon Nov 29 14:52:07 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Mon, 29 Nov 2010 13:52:07 +0000
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <id0ah2$2pr$1@dough.gmane.org>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
Message-ID: <AANLkTik1vGR8broTCQsMB0mXV0dBR4k9CSnKw8oW9N9y@mail.gmail.com>

Ah, does etree work in pypy?  That's just python right?

On Mon, Nov 29, 2010 at 1:40 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Amaury Forgeot d'Arc, 28.11.2010 11:44:
> > 2010/11/28 Maciej Fijalkowski
> >
> >> On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield wrote:
> >>> what xml libraries are people using with pypy?  What is working well?
> >>
> >> PyExpat works, although it's slow (ctypes-based implementation). I
> >> know genshi has some troubles with it, someone is debugging now.
> >> Besides I don't think there are any working (unless someone wrote a
> >> pure-python one)
> >
> > PyExpat is now a built-in module, implemented in RPython,
> > and should have reasonable performance.
>
> Hmm, reasonable?
>
> $ ./bin/pypy -m timeit -s 'import xml.etree.ElementTree as ET' \
>      'ET.parse("ot.xml")'
> 10 loops, best of 3: 1.27 sec per loop
>
> $ python2.7 -m timeit -s 'import xml.etree.ElementTree as ET' \
>      'ET.parse("ot.xml")'
> 10 loops, best of 3: 486 msec per loop
>
> $ python2.7 -m timeit -s 'import xml.etree.cElementTree as ET' \
>      'ET.parse("ot.xml")'
> 10 loops, best of 3: 33.7 msec per loop
>
>
> Stefan
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101129/c44423f2/attachment.html>

From wlavrijsen at lbl.gov  Mon Nov 29 14:55:48 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 29 Nov 2010 05:55:48 -0800 (PST)
Subject: [pypy-dev] double trouble with JIT
In-Reply-To: <4CF3AFC3.1040703@gmx.de>
References: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
	<4CF3A6A4.7080607@gmx.de>
	<alpine.LNX.2.00.1011290514450.2101@pc3-wlav.lbl.gov>
	<4CF3AFC3.1040703@gmx.de>
Message-ID: <alpine.LNX.2.00.1011290551560.2101@pc3-wlav.lbl.gov>

Hi Carl,

well, the real problem started when trying to use pythonify.py as that
gives me a segfault in:

Program received signal SIGSEGV, Segmentation fault.
0x0865b98c in pypy_g_BuiltinActivation_UwS_W_CPPOverload__run ()
(gdb) where
#0  0x0865b98c in pypy_g_BuiltinActivation_UwS_W_CPPOverload__run ()
#1  0xb7933fe4 in ?? ()
#2  0x00000000 in ?? ()

I then moved to the test and the segfault there (if the exception is not
seen) is the same. Hence I was hoping that the exception provided some
insight into the source of the segfault.

The benchmark uses .invoke(), so bypasses pythonify, but for a demo, I
prefer the syntax that pythonify offers

I'll do some more digging first.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From stefan_ml at behnel.de  Mon Nov 29 14:58:01 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 29 Nov 2010 14:58:01 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTik1vGR8broTCQsMB0mXV0dBR4k9CSnKw8oW9N9y@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTik1vGR8broTCQsMB0mXV0dBR4k9CSnKw8oW9N9y@mail.gmail.com>
Message-ID: <id0bha$7f1$1@dough.gmane.org>

Ren? Dudfield, 29.11.2010 14:52:
> On Mon, Nov 29, 2010 at 1:40 PM, Stefan Behnel wrote:
>> Amaury Forgeot d'Arc, 28.11.2010 11:44:
>>> 2010/11/28 Maciej Fijalkowski
>>>
>>>> On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield wrote:
>>>>> what xml libraries are people using with pypy?  What is working well?
>>>>
>>>> PyExpat works, although it's slow (ctypes-based implementation). I
>>>> know genshi has some troubles with it, someone is debugging now.
>>>> Besides I don't think there are any working (unless someone wrote a
>>>> pure-python one)
>>>
>>> PyExpat is now a built-in module, implemented in RPython,
>>> and should have reasonable performance.
>>
>> Hmm, reasonable?
>>
>> $ ./bin/pypy -m timeit -s 'import xml.etree.ElementTree as ET' \
>>       'ET.parse("ot.xml")'
>> 10 loops, best of 3: 1.27 sec per loop
>>
>> $ python2.7 -m timeit -s 'import xml.etree.ElementTree as ET' \
>>       'ET.parse("ot.xml")'
>> 10 loops, best of 3: 486 msec per loop
>>
>> $ python2.7 -m timeit -s 'import xml.etree.cElementTree as ET' \
>>       'ET.parse("ot.xml")'
>> 10 loops, best of 3: 33.7 msec per loop
>
> Ah, does etree work in pypy?  That's just python right?

Yes, ET is plain Python. cET is not, though, as the name indicates.

Stefan



From alex.nanou at gmail.com  Mon Nov 29 15:04:08 2010
From: alex.nanou at gmail.com (Alex A. Naanou)
Date: Mon, 29 Nov 2010 17:04:08 +0300
Subject: [pypy-dev] a possible leak in the object namespace...
Message-ID: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>

Hi All,

Decided to test the 1.4 with a couple of my usecases, got quite
surprising results...

To keep it short, I have several usecases that involve replacing the
object dictionary.

Here's a simple example:

---cut---

class D(dict):
	def __setitem__(self, key, value):
		print 'updating attr "%s" to "%s"...' % (key, value)
		super(D, self).__setitem__(key, value)

class X(object): pass

x = X()
x.__dict__ = D()

x.a = 1 # will print a nice log string...


# NOTE: this will not print anything in CPython (see P.S.)

--uncut--

With the release of version 1.4, I decided to test these usecases out
and benchmark them on PyPy and 15 minutes later I got results that
were surprising to say the least...

Expectations:
1) the normal/native namespace should have been a bit faster than the
hooked object on the first run. Both cases should have leveled to
about the same performance after the JIT finished it's job +/- a
constant.
2) all times should have been near constant.

What I got per point:
1) the object with native dict was slower by about three orders of
magnitude than the object with a hooked namespace.
2) sequential write benchmark runs on the normal object did not level
out, as they did with the hook, rather, they exhibited exponential
times (!!)


For details and actual test code see the attached file.




P.S. This specific functionality is a weak point in CPython for two reasons:
1) writing to .__dict__ demands a subclass of dict (not a real
problem, though over-restrictive in my view)
2) all interactions with the namespace are done directly via the
Python C-API, completely neglecting the high-level dict interface.


Thanks!

-- 
Alex.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: object_ns_hotwire.py
Type: application/octet-stream
Size: 2993 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101129/c2210c6b/attachment.obj>

From cfbolz at gmx.de  Mon Nov 29 19:46:11 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 29 Nov 2010 19:46:11 +0100
Subject: [pypy-dev] a possible leak in the object namespace...
In-Reply-To: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>
References: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>
Message-ID: <4CF3F4F3.9020204@gmx.de>

Hi Alex,

On 11/29/2010 03:04 PM, Alex A. Naanou wrote:
> With the release of version 1.4, I decided to test these usecases out
> and benchmark them on PyPy and 15 minutes later I got results that
> were surprising to say the least...
>
> Expectations:
> 1) the normal/native namespace should have been a bit faster than the
> hooked object on the first run. Both cases should have leveled to
> about the same performance after the JIT finished it's job +/- a
> constant.
> 2) all times should have been near constant.
>
> What I got per point:
> 1) the object with native dict was slower by about three orders of
> magnitude than the object with a hooked namespace.
> 2) sequential write benchmark runs on the normal object did not level
> out, as they did with the hook, rather, they exhibited exponential
> times (!!)

Don't do that then :-).


> For details and actual test code see the attached file.

The code you are trying is essentially this:

def test(obj, count=10000):
	t0 = time.time()
	for i in xrange(count):
		setattr(obj, 'a' + str(i), i)
	t1 = time.time()
	# return: total, time per write
	return t1 - t0, (t1 - t0)/count

This is not working very well with the non-overridden dict, because we 
don't optimize for this case at all. You are using

  a) lots of attributes, which we expect to be rare
  b) access them with setattr, which is a lot slower than a fixed attribute
  c) access a different attribute every loop iteration, which means the 
compiler has to produce one bit of code for every attribute

Read this, for some hints why this is the case:

http://morepypy.blogspot.com/2010/11/efficiently-implementing-python-objects.html

This is in theory fixable with enough work, but I am not sure that this 
is a common or useful use case. If you really need to do this, just use 
a normal dictionary. Or show me some real-world code that does this, and 
might think about the case some more.

Anyway, the timing behavior of the above loop is merely quadratic in the 
number of attributes, not exponential :-).

Cheers,

Carl Friedrich


From alex.nanou at gmail.com  Mon Nov 29 21:02:19 2010
From: alex.nanou at gmail.com (Alex A. Naanou)
Date: Mon, 29 Nov 2010 23:02:19 +0300
Subject: [pypy-dev] a possible leak in the object namespace...
In-Reply-To: <4CF3F4F3.9020204@gmx.de>
References: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>
	<4CF3F4F3.9020204@gmx.de>
Message-ID: <AANLkTin57JyAv1ihYa_FdCnwZqvDfDXbbuMDwvvp-a8f@mail.gmail.com>

On Mon, Nov 29, 2010 at 21:46, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Hi Alex,
>
> On 11/29/2010 03:04 PM, Alex A. Naanou wrote:
>> With the release of version 1.4, I decided to test these usecases out
>> and benchmark them on PyPy and 15 minutes later I got results that
>> were surprising to say the least...
>>
>> Expectations:
>> 1) the normal/native namespace should have been a bit faster than the
>> hooked object on the first run. Both cases should have leveled to
>> about the same performance after the JIT finished it's job +/- a
>> constant.
>> 2) all times should have been near constant.
>>
>> What I got per point:
>> 1) the object with native dict was slower by about three orders of
>> magnitude than the object with a hooked namespace.
>> 2) sequential write benchmark runs on the normal object did not level
>> out, as they did with the hook, rather, they exhibited exponential
>> times (!!)
>
> Don't do that then :-).

:)

>
>
>> For details and actual test code see the attached file.
>
> The code you are trying is essentially this:
>
> def test(obj, count=10000):
> ? ? ? ?t0 = time.time()
> ? ? ? ?for i in xrange(count):
> ? ? ? ? ? ? ? ?setattr(obj, 'a' + str(i), i)
> ? ? ? ?t1 = time.time()
> ? ? ? ?# return: total, time per write
> ? ? ? ?return t1 - t0, (t1 - t0)/count
>
> This is not working very well with the non-overridden dict, because we
> don't optimize for this case at all. You are using
>
> ?a) lots of attributes, which we expect to be rare
> ?b) access them with setattr, which is a lot slower than a fixed attribute
> ?c) access a different attribute every loop iteration, which means the
> compiler has to produce one bit of code for every attribute

This is intentional (all three points), I did not want the jit to
factor out the loop -- I wanted to time the initial attribute
creation...



>
> Read this, for some hints why this is the case:
>
> http://morepypy.blogspot.com/2010/11/efficiently-implementing-python-objects.html
>
> This is in theory fixable with enough work, but I am not sure that this
> is a common or useful use case. If you really need to do this, just use
> a normal dictionary. Or show me some real-world code that does this, and
> might think about the case some more.
>
> Anyway, the timing behavior of the above loop is merely quadratic in the
> number of attributes, not exponential :-).

Accepted, my mistake :)

But quadratic behaviour + a three orders of magnitude increase in time
it takes to create an attr is scarry... but you are right, how often
does that usecase happen? :)


Retested with:
  setattr(obj, 'a', i)

The results are *allot* better and it is indeed the common case :)



Thanks!

>
> Cheers,
>
> Carl Friedrich
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
Alex.


From alex.nanou at gmail.com  Mon Nov 29 21:12:49 2010
From: alex.nanou at gmail.com (Alex A. Naanou)
Date: Mon, 29 Nov 2010 23:12:49 +0300
Subject: [pypy-dev] a possible leak in the object namespace...
In-Reply-To: <AANLkTin57JyAv1ihYa_FdCnwZqvDfDXbbuMDwvvp-a8f@mail.gmail.com>
References: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>
	<4CF3F4F3.9020204@gmx.de>
	<AANLkTin57JyAv1ihYa_FdCnwZqvDfDXbbuMDwvvp-a8f@mail.gmail.com>
Message-ID: <AANLkTi=vMfpbunzsP9CtCvv9XPVXP6Zci_ar3EiXhMtw@mail.gmail.com>

On Mon, Nov 29, 2010 at 23:02, Alex A. Naanou <alex.nanou at gmail.com> wrote:
> On Mon, Nov 29, 2010 at 21:46, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
>> Hi Alex,
>>
>> On 11/29/2010 03:04 PM, Alex A. Naanou wrote:
>>> With the release of version 1.4, I decided to test these usecases out
>>> and benchmark them on PyPy and 15 minutes later I got results that
>>> were surprising to say the least...
>>>
>>> Expectations:
>>> 1) the normal/native namespace should have been a bit faster than the
>>> hooked object on the first run. Both cases should have leveled to
>>> about the same performance after the JIT finished it's job +/- a
>>> constant.
>>> 2) all times should have been near constant.
>>>
>>> What I got per point:
>>> 1) the object with native dict was slower by about three orders of
>>> magnitude than the object with a hooked namespace.
>>> 2) sequential write benchmark runs on the normal object did not level
>>> out, as they did with the hook, rather, they exhibited exponential
>>> times (!!)
>>
>> Don't do that then :-).
>
> :)
>
>>
>>
>>> For details and actual test code see the attached file.
>>
>> The code you are trying is essentially this:
>>
>> def test(obj, count=10000):
>> ? ? ? ?t0 = time.time()
>> ? ? ? ?for i in xrange(count):
>> ? ? ? ? ? ? ? ?setattr(obj, 'a' + str(i), i)
>> ? ? ? ?t1 = time.time()
>> ? ? ? ?# return: total, time per write
>> ? ? ? ?return t1 - t0, (t1 - t0)/count
>>
>> This is not working very well with the non-overridden dict, because we
>> don't optimize for this case at all. You are using
>>
>> ?a) lots of attributes, which we expect to be rare
>> ?b) access them with setattr, which is a lot slower than a fixed attribute
>> ?c) access a different attribute every loop iteration, which means the
>> compiler has to produce one bit of code for every attribute
>
> This is intentional (all three points), I did not want the jit to
> factor out the loop -- I wanted to time the initial attribute
> creation...
>
>
>
>>
>> Read this, for some hints why this is the case:
>>
>> http://morepypy.blogspot.com/2010/11/efficiently-implementing-python-objects.html
>>
>> This is in theory fixable with enough work, but I am not sure that this
>> is a common or useful use case. If you really need to do this, just use
>> a normal dictionary. Or show me some real-world code that does this, and
>> might think about the case some more.
>>
>> Anyway, the timing behavior of the above loop is merely quadratic in the
>> number of attributes, not exponential :-).
>
> Accepted, my mistake :)
>
> But quadratic behaviour + a three orders of magnitude increase in time
> it takes to create an attr is scarry... but you are right, how often
> does that usecase happen? :)
>
>
> Retested with:
> ?setattr(obj, 'a', i)
>
> The results are *allot* better and it is indeed the common case :)

come to think of it, the results appear to be too good, it looks like
the loop is optimized an only the last iteration is left in, so it's
not not representative...


>
>
>
> Thanks!
>
>>
>> Cheers,
>>
>> Carl Friedrich
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
>
> --
> Alex.
>



-- 
Alex.


From p.giarrusso at gmail.com  Mon Nov 29 21:54:24 2010
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Mon, 29 Nov 2010 21:54:24 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <id0ah2$2pr$1@dough.gmane.org>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
Message-ID: <AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>

On Mon, Nov 29, 2010 at 14:40, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Amaury Forgeot d'Arc, 28.11.2010 11:44:
>> 2010/11/28 Maciej Fijalkowski
>>
>>> On Sun, Nov 28, 2010 at 11:58 AM, Ren? Dudfield wrote:
>>>> what xml libraries are people using with pypy? ?What is working well?
>>>
>>> PyExpat works, although it's slow (ctypes-based implementation). I
>>> know genshi has some troubles with it, someone is debugging now.
>>> Besides I don't think there are any working (unless someone wrote a
>>> pure-python one)
>>
>> PyExpat is now a built-in module, implemented in RPython,
>> and should have reasonable performance.
>
> Hmm, reasonable?
>
> $ ./bin/pypy -m timeit -s 'import xml.etree.ElementTree as ET' \
> ? ? ?'ET.parse("ot.xml")'
> 10 loops, best of 3: 1.27 sec per loop
>
> $ python2.7 -m timeit -s 'import xml.etree.ElementTree as ET' \
> ? ? ?'ET.parse("ot.xml")'
> 10 loops, best of 3: 486 msec per loop
>
> $ python2.7 -m timeit -s 'import xml.etree.cElementTree as ET' \
> ? ? ?'ET.parse("ot.xml")'
> 10 loops, best of 3: 33.7 msec per loop

Is any JITting expected to trigger with so few iteractions? Or does
RPython saves the need for that? I tried increasing the loop count,
but I couldn't, because of two different bugs somewhere (in PyPy I
guess).

I tried ensuring that at least 1000 iterations were displayed, but
timeit doesn't work for more than 852 iterations on the attached
example (found on my HD):

$ pypy-trunk/pypy/translator/goal/pypy-c -m timeit -n 853 -s 'import
xml.etree.ElementTree as ET'      'ET.parse("extensionNames.xml")'
ImportError: No module named linecache

Now, even if linecache is imported locally, linecache.py exists
(located in the same path as timeit.py, i.e. lib-python/2.5.2/).

Furthermore, it works fine on the Python interpreter, suggesting that
the -m option might be part of the bug:
import timeit
a=timeit.Timer('ET.parse("extensionNames.xml")', 'import
xml.etree.ElementTree as ET')
a.timeit(1000)

However, a bigger timing count doesn't work:

>>>> a.timeit(10000)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/pgiarrusso/Documents/Research/Sorgenti/PyPy/pypy-trunk/lib-python/2.5.2/timeit.py",
line 161, in timeit
  File "<timeit-src>", line 6, in inner
  File "/Users/pgiarrusso/Documents/Research/Sorgenti/PyPy/pypy-trunk/lib_pypy/xml/etree/ElementTree.py",
line 862, in parse
  File "/Users/pgiarrusso/Documents/Research/Sorgenti/PyPy/pypy-trunk/lib_pypy/xml/etree/ElementTree.py",
line 579, in parse
IOError: [Errno 24] Too many open files: 'extensionNames.xml'

Inspection of the pypy process confirms a leak of file handles to the
XML files. Whether it is GC not being invoked, a missing destructor,
or simply because the code should release file handles, I dunno. Is
there a way to trigger explicit GC to workaround such issues?

Warning: all this is with a 32bit PyPy-1.4 on Mac OS X.

Bye
-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: extensionNames.xml
Type: text/xml
Size: 365 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101129/084bba6c/attachment.xml>

From amauryfa at gmail.com  Mon Nov 29 21:57:44 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 29 Nov 2010 21:57:44 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
Message-ID: <AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>

2010/11/29 Paolo Giarrusso <p.giarrusso at gmail.com>

> Inspection of the pypy process confirms a leak of file handles to the
> XML files. Whether it is GC not being invoked, a missing destructor,
> or simply because the code should release file handles, I dunno. Is
> there a way to trigger explicit GC to workaround such issues?
>

As usual:
    import gc
    gc.collect()

Calling gc.collect() is indeed a good idea if the code does not explicitly
close the files.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101129/7859c364/attachment.html>

From piotr.skamruk at gmail.com  Mon Nov 29 22:29:26 2010
From: piotr.skamruk at gmail.com (Piotr Skamruk)
Date: Mon, 29 Nov 2010 22:29:26 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
	<AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>
Message-ID: <AANLkTinZ8+0ona4nmHUBCSw-R1n4tawS+VrnSB+RFD_c@mail.gmail.com>

simplier would be set ulimit -n to 65536 (probably in /etc/security/limits.conf)

2010/11/29 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> 2010/11/29 Paolo Giarrusso <p.giarrusso at gmail.com>
>>
>> Inspection of the pypy process confirms a leak of file handles to the
>> XML files. Whether it is GC not being invoked, a missing destructor,
>> or simply because the code should release file handles, I dunno. Is
>> there a way to trigger explicit GC to workaround such issues?
>
> As usual:
> ?? ?import gc
> ?? ?gc.collect()
> Calling gc.collect() is indeed a good idea if the code does not explicitly
> close the files.
>
> --
> Amaury Forgeot d'Arc
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From p.giarrusso at gmail.com  Tue Nov 30 00:14:08 2010
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Tue, 30 Nov 2010 00:14:08 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTinZ8+0ona4nmHUBCSw-R1n4tawS+VrnSB+RFD_c@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
	<AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>
	<AANLkTinZ8+0ona4nmHUBCSw-R1n4tawS+VrnSB+RFD_c@mail.gmail.com>
Message-ID: <AANLkTi=AfnW9HagT2PaCPmCmrao0O=dNhq+XHvJEDat+@mail.gmail.com>

Hi all,
thanks to the tips, I verified on Mac OS X a 17% slowdown, after
manually taking the best times, vs Python-2.5 (32bit). Measuring on
the command line would give a 57% slowdown instead, because of lack of
warmup.
As a matter of fact, however, pyexpat is not involved here for PyPy,
and here (v1.4) it is still implemented through ctypes (in
lib_pypy/pyexpat.py), and not in RPython in pypy/rlib/.

Python 2.7 may well be faster, which might explain some extra
difference with Stefan's results.

It looks like the two bugs should be easy to fix:
- a file leak on the tested XML module, indeed
- an IOException on module opening converted to "file not found" - at
least in Java, file not found is a specific exception which can be
distinguished from generic I/O errors.

On Mon, Nov 29, 2010 at 22:29, Piotr Skamruk <piotr.skamruk at gmail.com> wrote:
> simplier would be set ulimit -n to 65536 (probably in /etc/security/limits.conf)

Thanks, I needed both this and the GC tips, since during a test run to
run 10^4 iterations, I can't call the GC and still get meaningful
results.

[I'm on Mac OS X though, so ulimit -S -n 10240 is the best one can do,
otherwise "Invalid argument", i.e. EINVAL, results].

Additionally, I just discovered that the ImportError on "import
linecache"  looks filehandle-related as well, because changing the
ulimit changes the iteration count triggering the error, so it's
likely an effect of the same bug. Still, the original error message
should be preserved, and this should be easy to fix.

In these conditions, my best results after warming up are:

0.358 ms PyPy-JIT-32bit (see below for JIT logs)
0.305 ms CPython-2.5-32bit
0.269 ms CPython-2.6-64bit
0.553 ms PyPy-64bit-noJIT, rev 79307, 21 Nov 2010

which means a 17% slowdown on comparable setups, rather than a 2x
slowdown; measuring with timeit on the cmd line, instead, would give a
57% slowdown.
All this is on a very small input file, the one I attached before.

That's for the total of 1000 iterations, on a Core 2 Duo 2.6GHz.

I don't report the average because:
a) it is difficult to get something significant anyway (I don't want
to code confidence intervals, and automated tools wouldn't call GC
appropriately)
b) I expect the deviation to be due more to unrelated load on my
laptop (around 12-18% CPU) than to actual spread of the runtime.

I set PYPYLOG='jit-summary:-' before the PyPy-JIT run and got this - I
hope somebody can check from this whether the JIT is working
successfully.

[f2dd1fbaa1c2] {jit-summary
Tracing:        25      0.163456
Backend:        23      0.017392
Running asm:            191214
Blackhole:              2012
TOTAL:                  502.543032
ops:                    68338
recorded ops:           32764
  calls:                1759
guards:                 18005
opt ops:                2757
opt guards:             696
forcings:               111
abort: trace too long:  2
abort: compiling:       0
abort: vable escape:    0
nvirtuals:              6693
nvholes:                1059
nvreused:               3979
Total # of loops:       18
Total # of bridges:     6
Freed # of loops:       0
Freed # of bridges:     0
[f2dd1fc141a8] jit-summary}

Best regards.

> 2010/11/29 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
>> 2010/11/29 Paolo Giarrusso <p.giarrusso at gmail.com>
>>>
>>> Inspection of the pypy process confirms a leak of file handles to the
>>> XML files. Whether it is GC not being invoked, a missing destructor,
>>> or simply because the code should release file handles, I dunno. Is
>>> there a way to trigger explicit GC to workaround such issues?
>>
>> As usual:
>> ?? ?import gc
>> ?? ?gc.collect()
>> Calling gc.collect() is indeed a good idea if the code does not explicitly
>> close the files.

-- 
Paolo Giarrusso - Ph.D. Student
http://www.informatik.uni-marburg.de/~pgiarrusso/


From amauryfa at gmail.com  Tue Nov 30 00:45:05 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 30 Nov 2010 00:45:05 +0100
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTi=AfnW9HagT2PaCPmCmrao0O=dNhq+XHvJEDat+@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
	<AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>
	<AANLkTinZ8+0ona4nmHUBCSw-R1n4tawS+VrnSB+RFD_c@mail.gmail.com>
	<AANLkTi=AfnW9HagT2PaCPmCmrao0O=dNhq+XHvJEDat+@mail.gmail.com>
Message-ID: <AANLkTi=QPWdXrCJGMRNA-1NMnNT9m22cPBDmoBEH8eWX@mail.gmail.com>

2010/11/30 Paolo Giarrusso <p.giarrusso at gmail.com>

> As a matter of fact, however, pyexpat is not involved here for PyPy,
> and here (v1.4) it is still implemented through ctypes (in
> lib_pypy/pyexpat.py), and not in RPython in pypy/rlib/.
>

Did you compile pypy yourself?
if the expat development files are present, the translation should build the
pyexpat module:

Python 2.5.2 (79656, Nov 29 2010, 21:05:28)
[PyPy 1.4.0] on linux2
>>>> import pyexpat
>>>> pyexpat
<module 'pyexpat' (built-in)>

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101130/661c0e3e/attachment.html>

From drsalists at gmail.com  Tue Nov 30 02:28:20 2010
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 29 Nov 2010 17:28:20 -0800
Subject: [pypy-dev] gdbm
In-Reply-To: <4CF36B99.9080104@gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>
	<amauryfa@gmail.com>
	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>
	<201011160222.oAG2Mrfv004177@theraft.openend.se>
	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>
	<4CF03ED2.3050801@gmail.com>
	<AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>
	<AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>
	<AANLkTincyJPhS_wm92eD3fTMDJtcqJ0ruSEytm6yKZ2W@mail.gmail.com>
	<4CF36B99.9080104@gmail.com>
Message-ID: <AANLkTikCH8Bezf+uxSijVzy14PTKKHYph5=QKGMcHUDx@mail.gmail.com>

On Mon, Nov 29, 2010 at 1:00 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> On 27/11/10 04:07, Dan Stromberg wrote:
>
> > This module can be used with the ?classic? ndbm interface, the BSD DB
> > compatibility interface, or the GNU GDBM compatibility interface. On
> Unix, the
> > *configure* script will attempt to locate the appropriate header file to
> > simplify building this module.
> >
> > I suppose that means that if it can't find ndbm (which at one time was
> hard
> > due to licensing, but last I heard it'd become readily available), it's
> free
> > to pretend it has ndbm using something else.
> >
> > I'd call that puzzlingly worded - it's not the interface that's changing,
> but
> > the backend implementation.  But perhaps dbm.py is free to use Berkeley
> DB if
> > it prefers to pretend it can never find ndbm.  And perhaps I shouldn't
> have
> > skimmed that page so quickly ^_^
> >
> > CPython 2.7's configure script has:
> >
> >   --with-dbmliborder=db1:db2:...
> >                           order to check db backends for dbm. Valid value
> is a
> >                           colon separated string with the backend names
> >                           `ndbm', `gdbm' and `bdb'.
> >
> >
>
> so, having a dbm.py which links to libdb is fine, and it's also what you
> get
> with cpython on ubuntu.  There is still the issue of how to find the
> correct
> library name, as it seems it can vary (it was db-4.5 when the module was
> written, it's db-4.8 nowadays), but this is a bit orthogonal to what we are
> discussing here.
>
> To summarize, I think we should keep the current dbm.py to link against
> libdb,
> and integrate your gdbm.py to link against libgdbm.  But before merging it
> to
> trunk, I'd like to solve the issue of code duplication between the two
> modules.
>
> ciao,
> Anto
>

Agreed.

I should have time for this sometime this week or the next.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101129/5bcc967c/attachment.html>

From wlavrijsen at lbl.gov  Tue Nov 30 05:35:06 2010
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 29 Nov 2010 20:35:06 -0800 (PST)
Subject: [pypy-dev] double trouble with JIT
In-Reply-To: <4CF3AFC3.1040703@gmx.de>
References: <alpine.LNX.2.00.1011290502270.2101@pc3-wlav.lbl.gov>
	<4CF3A6A4.7080607@gmx.de>
	<alpine.LNX.2.00.1011290514450.2101@pc3-wlav.lbl.gov>
	<4CF3AFC3.1040703@gmx.de>
Message-ID: <alpine.LNX.2.00.1011292033090.10976@pc3-wlav.lbl.gov>

Hi Carl,

> Given that we are not running PyPy's test suite on PyPy nightly currently, I 
> think it would be safer to keep running the tests on top of CPython.

of course, that begs the question how one tests pypy? I'm thinking of making
the benchmarks more elaborate and use those as my post-translation tests.

Btw., I solved the original segfault that I was after, but that didn't change
anything for the particular error in this e-mail thread, though.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net


From fijall at gmail.com  Tue Nov 30 08:13:41 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 30 Nov 2010 09:13:41 +0200
Subject: [pypy-dev] which xml libraries? was (Re: PyPy 1.4 released)
In-Reply-To: <AANLkTi=QPWdXrCJGMRNA-1NMnNT9m22cPBDmoBEH8eWX@mail.gmail.com>
References: <AANLkTikdiugoQSDDSZOSoYL4jERTL1i_MPCuBT6gkTQx@mail.gmail.com>
	<AANLkTik-n2amKz=z9fgFtqxXNPDJMNh0fKOg7vnwGcTm@mail.gmail.com>
	<AANLkTimFFHWSwYYVNGouYc=_SdH235Q60YX-vvcmH6o6@mail.gmail.com>
	<id0ah2$2pr$1@dough.gmane.org>
	<AANLkTimkE12+bGDR7fjzjgeAVA43h07-ZLwFSWmSLuvC@mail.gmail.com>
	<AANLkTi=+bWFGZiaccZX8SGVCTzNGfVSbCVf5fqhGYN08@mail.gmail.com>
	<AANLkTinZ8+0ona4nmHUBCSw-R1n4tawS+VrnSB+RFD_c@mail.gmail.com>
	<AANLkTi=AfnW9HagT2PaCPmCmrao0O=dNhq+XHvJEDat+@mail.gmail.com>
	<AANLkTi=QPWdXrCJGMRNA-1NMnNT9m22cPBDmoBEH8eWX@mail.gmail.com>
Message-ID: <AANLkTi=tNGbymFodLQgyB04zBf3FZj1k0dW3GhNCu9HM@mail.gmail.com>

On Tue, Nov 30, 2010 at 1:45 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> 2010/11/30 Paolo Giarrusso <p.giarrusso at gmail.com>
>>
>> As a matter of fact, however, pyexpat is not involved here for PyPy,
>> and here (v1.4) it is still implemented through ctypes (in
>> lib_pypy/pyexpat.py), and not in RPython in pypy/rlib/.

It's also module/pyexpat and not rlib (rlib is for RPython libraries)

>
> Did you compile pypy yourself?
> if the expat development files are present, the translation should build the
> pyexpat module:
> Python 2.5.2 (79656, Nov 29 2010, 21:05:28)
> [PyPy 1.4.0] on linux2
>>>>> import pyexpat
>>>>> pyexpat
> <module 'pyexpat' (built-in)>
> --
> Amaury Forgeot d'Arc
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>


From anto.cuni at gmail.com  Tue Nov 30 08:19:21 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 30 Nov 2010 08:19:21 +0100
Subject: [pypy-dev] gdbm
In-Reply-To: <AANLkTikCH8Bezf+uxSijVzy14PTKKHYph5=QKGMcHUDx@mail.gmail.com>
References: <AANLkTik+KkKus5ztPZVMmJUCe7vWZU6uU4Tv==5neeJa@mail.gmail.com>	<amauryfa@gmail.com>	<AANLkTinw7MH7yOkK5-yYEy64ozjvsVje7VuMFfq6t7OT@mail.gmail.com>	<201011160222.oAG2Mrfv004177@theraft.openend.se>	<AANLkTi=0oPjxZ0WktVB=An4TtfRBR7jLfrLrERMox_v+@mail.gmail.com>	<4CF03ED2.3050801@gmail.com>	<AANLkTikDXB1xD=+aF1bxNAT3VE4+NBarEXws20NOn=PP@mail.gmail.com>	<AANLkTinLumxq52qGsjCRfkcYAGURxzwGXaQv_Z0mVHZB@mail.gmail.com>	<AANLkTincyJPhS_wm92eD3fTMDJtcqJ0ruSEytm6yKZ2W@mail.gmail.com>	<4CF36B99.9080104@gmail.com>
	<AANLkTikCH8Bezf+uxSijVzy14PTKKHYph5=QKGMcHUDx@mail.gmail.com>
Message-ID: <4CF4A579.3060208@gmail.com>

On 30/11/10 02:28, Dan Stromberg wrote:

> Agreed.
> 
> I should have time for this sometime this week or the next.

thank you! :-)


From arigo at tunes.org  Tue Nov 30 11:04:20 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 30 Nov 2010 11:04:20 +0100
Subject: [pypy-dev] PyPy-JIT memory usage compared to CPython?
In-Reply-To: <504667.29687.qm@web111301.mail.gq1.yahoo.com>
References: <AANLkTi=HSn6x127NyDV5WdQD=-k4eh0Gmoo5ht6vXeVi@mail.gmail.com>
	<504667.29687.qm@web111301.mail.gq1.yahoo.com>
Message-ID: <AANLkTinSMNXEo4jxe4C5q2PG76RjDLaFQC0jCs_ShhRQ@mail.gmail.com>

Hi,

On Tue, Nov 30, 2010 at 10:58 AM, Andy <angelflow at yahoo.com> wrote:
> Can you explain a bit about what did you mean by "program contains mainly instances"? And what is __slots__?

I meant basically: try with your particular use case, because it is
hard to give one answer that fits all cases.


Armin


From angelflow at yahoo.com  Tue Nov 30 10:58:13 2010
From: angelflow at yahoo.com (Andy)
Date: Tue, 30 Nov 2010 01:58:13 -0800 (PST)
Subject: [pypy-dev] PyPy-JIT memory usage compared to CPython?
In-Reply-To: <AANLkTi=HSn6x127NyDV5WdQD=-k4eh0Gmoo5ht6vXeVi@mail.gmail.com>
Message-ID: <504667.29687.qm@web111301.mail.gq1.yahoo.com>

Can you explain a bit about what did you mean by "program contains mainly instances"? And what is __slots__?

Thanks

--- On Mon, 11/29/10, Armin Rigo <arigo at tunes.org> wrote:

> From: Armin Rigo <arigo at tunes.org>
> Subject: Re: [pypy-dev] PyPy-JIT memory usage compared to CPython?
> To: "Andy" <angelflow at yahoo.com>
> Cc: pypy-dev at codespeak.net
> Date: Monday, November 29, 2010, 5:32 AM
> Hi,
> 
> On Sat, Nov 27, 2010 at 12:03 AM, Andy <angelflow at yahoo.com>
> wrote:
> > I'm particularly interested in the memory usage of
> PyPy-JIT vs. CPython when running Django. In general would
> PyPy-JIT increase or decrease memory usage?
> 
> It depends on the kind of application.? PyPy with a
> JIT may get to
> 1.5x or 2x more than CPython, but it may also be smaller
> than CPython
> if your program contains mainly instances and you didn't
> feel like
> carefully putting __slots__ everywhere.
> 
> 
> A bient?t,
> 
> Armin.
> 


      


From marius at pov.lt  Tue Nov 30 12:42:05 2010
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 30 Nov 2010 13:42:05 +0200
Subject: [pypy-dev] PyPy 1.4 released
In-Reply-To: <4CF2389B.90801@gmail.com>
References: <AANLkTi=D-NQgKfbcGDj5btLJuffXx7fY8S0FLoa9ufK0@mail.gmail.com>
	<AANLkTik92OmCMedjNyZu7635k-GgJVXdLuzejO+VPR-g@mail.gmail.com>
	<AANLkTik_J3Ccosf26af5WiCv3BKn9hEFbwfqKPzebpu8@mail.gmail.com>
	<AANLkTikRoryqcZnfdHrntxf7gxo7VBbzaKjgLOt0ycSD@mail.gmail.com>
	<AANLkTi=4ZOuzQiGiNJ4+8db+mYSbJ0+M4VdSDh+QmiHc@mail.gmail.com>
	<AANLkTinYsa_5F4+9b95JrWaJXibxt61iym2-UPEBPpEj@mail.gmail.com>
	<4CF0A297.9090202@gmail.com>
	<AANLkTi=BYi2ohGbdkeSfRo6HPDp665r0_yv2P1uqRnLB@mail.gmail.com>
	<AANLkTimuhqCt5WkgcrqovR1gHN09LrUz+ZAsNc8czUjP@mail.gmail.com>
	<4CF2389B.90801@gmail.com>
Message-ID: <20101130114204.GA30038@fridge.pov.lt>

On Sun, Nov 28, 2010 at 12:10:19PM +0100, Antonio Cuni wrote:
> On 28/11/10 10:48, Maciej Fijalkowski wrote:
> >> > i got python-magic working , after i installed without easy_install
> >> > (easy_install fail because it tried to install ctypes).
> 
> > great
> 
> well, that's still a bit strange. Why does easy_install try to install ctypes,
> given that it's in the stdlib?

The standard library doesn't have setuptools metadata that would tell
easy_install about ctypes already being available, so easy_install tries
to download and install ctypes if asked to.

Marius Gedminas
-- 
I'm a shareware signature!  Send $2 if you use me, $10 for a manual.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101130/1077eaf0/attachment.pgp>

From cfbolz at gmx.de  Tue Nov 30 18:33:45 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Tue, 30 Nov 2010 18:33:45 +0100
Subject: [pypy-dev] a possible leak in the object namespace...
In-Reply-To: <AANLkTin57JyAv1ihYa_FdCnwZqvDfDXbbuMDwvvp-a8f@mail.gmail.com>
References: <AANLkTi=+oQzRR=gnvGhHij8+RJkybD2MwgV6NPJd54rJ@mail.gmail.com>
	<4CF3F4F3.9020204@gmx.de>
	<AANLkTin57JyAv1ihYa_FdCnwZqvDfDXbbuMDwvvp-a8f@mail.gmail.com>
Message-ID: <4CF53579.6080504@gmx.de>

Hi Alex,

On 11/29/2010 09:02 PM, Alex A. Naanou wrote:
> On Mon, Nov 29, 2010 at 21:46, Carl Friedrich Bolz<cfbolz at gmx.de>  wrote:
[snip]
>>   a) lots of attributes, which we expect to be rare
>>   b) access them with setattr, which is a lot slower than a fixed attribute
>>   c) access a different attribute every loop iteration, which means the
>> compiler has to produce one bit of code for every attribute
>
> This is intentional (all three points), I did not want the jit to
> factor out the loop -- I wanted to time the initial attribute
> creation...

Yes, I fear initial attribute creation is never going to be very efficient.

[snip]
>
> But quadratic behaviour + a three orders of magnitude increase in time
> it takes to create an attr is scarry... but you are right, how often
> does that usecase happen? :)

I improved some things so that setattr/getattr is a bit faster. Should 
now only be one or two orders of magnitude :-). If you run it tomorrow 
with the nightly build from tonight, you should see an improvement.
The quadraticness is not easily fixable without giving up all 
optimizations with instances of more than X (maybe 1000?) attributes. 
Again, I don't think this is common. And I don't want to chose an X.

Cheers,

Carl Friedrich


From sasmekoll at gmail.com  Wed Nov 24 21:41:53 2010
From: sasmekoll at gmail.com (Vovan)
Date: Wed, 24 Nov 2010 23:41:53 +0300
Subject: [pypy-dev] =?utf-8?b?0J/QuNC60LDQvw==?=
Message-ID: <4cf7a5b5.960acc0a.33d1.2224@mx.google.com>

?????????? ????????  ???????? ???? ???????????? ?????????????????????????? ???????????? ?? ?? ??????????????, ?????????????? ???????????????? ???????????????? ?? ???????? ???????????????? ???? ???????????? ?? ?????????????????????? ?????????? ???????? ???????????? ???? ???????????? ?????? ?????????????????????????? ?? ????????????????? , ?? ???????????? ?????????????????? ??????????????, ?????????? ???????????? ?????????????? ?????????????? ?????? ?????????? ???????????????????? ??????????????????!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/1f05085d/attachment.html>

From sasmekoll at gmail.com  Wed Nov 24 21:44:49 2010
From: sasmekoll at gmail.com (Vovan)
Date: Wed, 24 Nov 2010 23:44:49 +0300
Subject: [pypy-dev] =?utf-8?b?0J/QuNC60LDQvw==?=
Message-ID: <4cf7a649.cc97cc0a.16f2.2037@mx.google.com>

?????????? ????????  ???????? ???? ???????????? ?????????????????????????? ???????????? ?? ?? ??????????????, ?????????????? ???????????????? ???????????????? ?? ???????? ???????????????? ???? ???????????? ?? ?????????????????????? ?????????? ???????? ???????????? ???? ???????????? ?????? ?????????????????????????? ?? ????????????????? , ?? ???????????? ?????????????????? ??????????????, ?????????? ???????????? ?????????????? ?????????????? ?????? ?????????? ???????????????????? ??????????????????!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/49bcdcae/attachment.html>

From sasmekoll at gmail.com  Wed Nov 24 21:45:46 2010
From: sasmekoll at gmail.com (Vovan)
Date: Wed, 24 Nov 2010 23:45:46 +0300
Subject: [pypy-dev] =?utf-8?b?0J/QuNC60LDQvw==?=
Message-ID: <4cf7a67a.4402cc0a.48f5.2225@mx.google.com>

?????????? ????????  ???????? ???? ???????????? ?????????????????????????? ???????????? ?? ?? ??????????????, ?????????????? ???????????????? ???????????????? ?? ???????? ???????????????? ???? ???????????? ?? ?????????????????????? ?????????? ???????? ???????????? ???? ???????????? ?????? ?????????????????????????? ?? ????????????????? , ?? ???????????? ?????????????????? ??????????????, ?????????? ???????????? ?????????????? ?????????????? ?????? ?????????? ???????????????????? ??????????????????!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/ea13295b/attachment.html>

From sasmekoll at gmail.com  Wed Nov 24 21:42:51 2010
From: sasmekoll at gmail.com (Vovan)
Date: Wed, 24 Nov 2010 23:42:51 +0300
Subject: [pypy-dev] =?utf-8?b?0J/QuNC60LDQvw==?=
Message-ID: <4cf7a5e5.948fcc0a.5118.2184@mx.google.com>

?????????? ????????  ???????? ???? ???????????? ?????????????????????????? ???????????? ?? ?? ??????????????, ?????????????? ???????????????? ???????????????? ?? ???????? ???????????????? ???? ???????????? ?? ?????????????????????? ?????????? ???????? ???????????? ???? ???????????? ?????? ?????????????????????????? ?? ????????????????? , ?? ???????????? ?????????????????? ??????????????, ?????????? ???????????? ?????????????? ?????????????? ?????? ?????????? ???????????????????? ??????????????????!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/b149a7c0/attachment.html>

From sasmekoll at gmail.com  Wed Nov 24 21:43:49 2010
From: sasmekoll at gmail.com (Vovan)
Date: Wed, 24 Nov 2010 23:43:49 +0300
Subject: [pypy-dev] =?utf-8?b?0J/QuNC60LDQvw==?=
Message-ID: <4cf7a616.948fcc0a.34d7.215c@mx.google.com>

?????????? ????????  ???????? ???? ???????????? ?????????????????????????? ???????????? ?? ?? ??????????????, ?????????????? ???????????????? ???????????????? ?? ???????? ???????????????? ???? ???????????? ?? ?????????????????????? ?????????? ???????? ???????????? ???? ???????????? ?????? ?????????????????????????? ?? ????????????????? , ?? ???????????? ?????????????????? ??????????????, ?????????? ???????????? ?????????????? ?????????????? ?????? ?????????? ???????????????????? ??????????????????!!!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101124/c3df3d94/attachment.html>

From sasmekoll at gmail.com  Thu Nov 25 02:21:36 2010
From: sasmekoll at gmail.com (Vovan)
Date: Thu, 25 Nov 2010 04:21:36 +0300
Subject: [pypy-dev] =?utf-8?b?0JbQtdC90YHQutCw0Y8g0LvQvtCz0LjQutCw?=
Message-ID: <4cf7e88f.5989cc0a.4c3c.34f2@mx.google.com>

???????????? ???????????????? ????????????? ?????????? ???????????????? ???? ??????????, ?????????? ?????? ???????????????? ???????????????? ???? ???? ???????????????????? ???????????????????????? ????????????? ???? ???????????????????????? ?????????????????? ???????????????? ?????????????????? ???? ?? ???????????????? ???????????????? ?????????????????? ???????? ?????????????????????????????????? ?? ???????????????????? ???????????????? ?????? ????????????.
http://samec.org.ua/?p=202
???? ????????????? ????????????????????, ?????? ????????????????????. ?????????? ?????? ?????????????? ?????????????? ??????????????.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20101125/89134871/attachment.html>