From arigo at tunes.org  Fri Mar  1 11:34:14 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Mar 2013 11:34:14 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
Message-ID: <CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>

Hi Roger,

On Thu, Feb 28, 2013 at 6:00 PM, Roger Flores <aidembb at yahoo.com> wrote:
> OK then. Unzip it, grab a text file large enough to warm up the jit, and run
> the line to generate the log for jitviewer.

I think there is nothing quite broken for PyPy.  It just has a very
long warm-up time.  On my 64-bit laptop, it runs frank.txt in 14+10
seconds.  If I replace frank.txt by 5 concatenated copies of it, it
takes 34+29 seconds.  That means every additional run takes only 5
seconds.  For comparison the CPython time, also on 64-bit, is 21+21
seconds for frank.txt.  It's not very clear why, but the warm-up time
can be much smaller or bigger depending on the example; it's some
current and future research to improve on that aspect.

On 32-bit there is extra time needed --- both on PyPy and on CPython
--- because the numbers you use overflow signed 32-bit ints, and it
needs longs.  On PyPy we could in theory improve for that case, e.g.
by writing an alternate implementation for longs, for example for
numbers that fit into two regular-sized integers.  (There is actually
one implementation for that, but it's not complete and limited so far,
so not enabled by default; see pypy.objspace.std.smalllongobject.py.
A more complete version would really use two integers, rather than a
"long long" integer.)


A bient?t,

Armin.

From arigo at tunes.org  Fri Mar  1 11:54:09 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Mar 2013 11:54:09 +0100
Subject: [pypy-dev] curses_cffi and others
Message-ID: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>

Hi all, hi Jeremy,

I'm thinking about merging the curses_cffi branch, and I see efforts
going on in sqlite-cffi.  Great :-)

We need to think about how we want to support cffi-based modules in
PyPy.  So far the idea is to include the official release of CFFI
together with regular PyPy installations.  Then, how can we disable
"pip install cffi" from breaking everything up by installing a
different release of cffi on top of PyPy?  Is it enough to detect in
cffi's setup.py "oh, it's PyPy and cffi is already installed, so I'll
do nothing"?  Should it still check that the version number matches,
and complain if it doesn't?


A bient?t,

Armin.

From kostia.lopuhin at gmail.com  Fri Mar  1 12:28:28 2013
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Fri, 1 Mar 2013 15:28:28 +0400
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
Message-ID: <CAAvCB=TKAo8QxyX59varNWdcX0kQE_ysQ+TKgj7AqcE4MzMJiw@mail.gmail.com>

Hi!
Maybe a possible way is that you have "major" releases of cffi without
backward incompatible changes that correspond to PyPy releases - so
that any cffi 1.x works on PyPy 2.0, and any cffi 2.x works on PyPy
2.1, etc. And than you can complain if there is major version mismatch
of cffi.
So there will be two cffi branches active at any time - the one that
works with released PyPy, and the one that works with nightly.

Thank you for cffi :)

2013/3/1 Armin Rigo <arigo at tunes.org>:
> Hi all, hi Jeremy,
>
> I'm thinking about merging the curses_cffi branch, and I see efforts
> going on in sqlite-cffi.  Great :-)
>
> We need to think about how we want to support cffi-based modules in
> PyPy.  So far the idea is to include the official release of CFFI
> together with regular PyPy installations.  Then, how can we disable
> "pip install cffi" from breaking everything up by installing a
> different release of cffi on top of PyPy?  Is it enough to detect in
> cffi's setup.py "oh, it's PyPy and cffi is already installed, so I'll
> do nothing"?  Should it still check that the version number matches,
> and complain if it doesn't?
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From stefano at rivera.za.net  Fri Mar  1 12:30:18 2013
From: stefano at rivera.za.net (Stefano Rivera)
Date: Fri, 1 Mar 2013 13:30:18 +0200
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
Message-ID: <20130301113018.GK29115@bach.rivera.co.za>

Hi Armin (2013.03.01_12:54:09_+0200)
> We need to think about how we want to support cffi-based modules in
> PyPy.  So far the idea is to include the official release of CFFI
> together with regular PyPy installations.

The related question is: How do we ship CFFI modules in PyPy? Import all
the cffi modules once after translation to produce the cached shared
libraries?

> Then, how can we disable "pip install cffi" from breaking everything
> up by installing a different release of cffi on top of PyPy?  Is it
> enough to detect in cffi's setup.py "oh, it's PyPy and cffi is already
> installed, so I'll do nothing"?

That sounds reasonable. I guess it should print something about that and
exit 0.

> Should it still check that the version number matches, and complain if
> it doesn't?

Is there going to be a backend API version that's independent of the
cffi version?

At any rate, reporting that the backend is incompatible with the
requested cffi seems reasonable.

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

From matti.picus at gmail.com  Fri Mar  1 15:47:37 2013
From: matti.picus at gmail.com (matti picus)
Date: Fri, 1 Mar 2013 16:47:37 +0200
Subject: [pypy-dev] test_ztranslation in micronumpy/test not tranlating all
 ufunc functions in types.py
Message-ID: <CAO8bCgi_x7UGuwxAgirJu6EaLQM1rkKM2nxjJijvMAKsvfNtQw@mail.gmail.com>

test/test_ztranslation still passes even if I put an xxx in
Bool.reciprocal (line 397) in micronumpy/types.py. What am I doing
wrong?
Matti

From g2p.code at gmail.com  Fri Mar  1 16:22:55 2013
From: g2p.code at gmail.com (Gabriel de Perthuis)
Date: Fri, 1 Mar 2013 15:22:55 +0000 (UTC)
Subject: [pypy-dev] curses_cffi and others
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
Message-ID: <kgqh4f$ovd$1@ger.gmane.org>

> Hi all, hi Jeremy,
> 
> I'm thinking about merging the curses_cffi branch, and I see efforts
> going on in sqlite-cffi.  Great :-)
> 
> We need to think about how we want to support cffi-based modules in
> PyPy.  So far the idea is to include the official release of CFFI
> together with regular PyPy installations.  Then, how can we disable
> "pip install cffi" from breaking everything up by installing a
> different release of cffi on top of PyPy?  Is it enough to detect in
> cffi's setup.py "oh, it's PyPy and cffi is already installed, so I'll
> do nothing"?  Should it still check that the version number matches,
> and complain if it doesn't?

Hello,
Packages using CFFI should be able to install-require CFFI whatever the
interpreter is, including requiring newer versions than the one the interpreter
ships. (I can't require users of my packages to install a newer interpreter,
or wait for a given CFFI version to land in a CPython release, whereas setup
requirements basically come for free).

Giving the embedded copy a different name should prevent most kinds of
interference (it's the way Django, requests, etc use). So call the embedded
CFFI something like cffi_embedded. That way a project can require a newer CFFI
from PyPI without the two of them interfering. _cffi_backend (the one with
special interpreter support on PyPy) can be shared between both copies, but
it's smaller and easier to maintain compatibility.

Incidentally, trying to share an import name between two packages can't
be done reliably (setuptools used to do some path hacks to put eggs earlier
in the import path, but they were fragile and broke with virtualenv; plain
distutils doesn't have those hacks). Explicitly selecting implementations
with `import as` and an ImportError clause is much more predictable.


From arigo at tunes.org  Fri Mar  1 17:30:14 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Mar 2013 17:30:14 +0100
Subject: [pypy-dev] test_ztranslation in micronumpy/test not tranlating
 all ufunc functions in types.py
In-Reply-To: <CAO8bCgi_x7UGuwxAgirJu6EaLQM1rkKM2nxjJijvMAKsvfNtQw@mail.gmail.com>
References: <CAO8bCgi_x7UGuwxAgirJu6EaLQM1rkKM2nxjJijvMAKsvfNtQw@mail.gmail.com>
Message-ID: <CAMSv6X3YUNBBgM3b87E7y8kZuq_wSrpSC05wnGK8Jo-X_1GTbg@mail.gmail.com>

Hi Matti,

On Fri, Mar 1, 2013 at 3:47 PM, matti picus <matti.picus at gmail.com> wrote:
> test/test_ztranslation still passes even if I put an xxx in
> Bool.reciprocal (line 397) in micronumpy/types.py. What am I doing
> wrong?

Fixed in e61ccf099ed1.  That was really a bug.


A bient?t,

Armin.

From arigo at tunes.org  Fri Mar  1 17:44:27 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Mar 2013 17:44:27 +0100
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <kgqh4f$ovd$1@ger.gmane.org>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
	<kgqh4f$ovd$1@ger.gmane.org>
Message-ID: <CAMSv6X0n3f+oq-18fqFgq1ntPy=so__UcaxP08A6t15T3Piyyg@mail.gmail.com>

Hi Gabriel,

On Fri, Mar 1, 2013 at 4:22 PM, Gabriel de Perthuis <g2p.code at gmail.com> wrote:
> Packages using CFFI should be able to install-require CFFI whatever the
> interpreter is, including requiring newer versions than the one the interpreter
> ships.

The problem is that the pure Python CFFI is rather tied to the
extension module _cffi_backend (e.g. you can't add new methods to
cdata objects from the pure Python side).

So the problem is really: it's not possible to install a newer CFFI on
top of an existing PyPy interpreter at all (unless you install a newer
_cffi_backend.so as a cpyext C extension module, which completely
defeats the performance point).  But the idea is that CFFI should soon
be more or less frozen; I don't want to keep growing its feature set
forever.

Even so, more generally, I think that requiring users to install a
newer version of the interpreter is actually feasible, when we are
talking about PyPy and not CPython.  A "newer version" of PyPy is just
an internal improvement, but not a language change --- it's always
2.7.  I think that so far, our users are (or should be) ready to deal
with this minor annoyance; if they are not and really want a
completely stable, never-changing interpreter, then they're going to
stick with CPython anyway.

In other words, I regard this as similar to: "this program requires
PyPy 2.0 and not a lower version because otherwise its usage of
greenlets is going to be a huge performance hit".


A bient?t,

Armin.

From aidembb at yahoo.com  Fri Mar  1 22:13:23 2013
From: aidembb at yahoo.com (Roger Flores)
Date: Fri, 1 Mar 2013 13:13:23 -0800 (PST)
Subject: [pypy-dev] Slow int code
In-Reply-To: <CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
Message-ID: <1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>



>I think there is nothing quite broken for PyPy.? It just has a very long warm-up time.
I think the jit has warmed up, for a few different reasons.? Maybe some of them are leading me astray though.



>That means every additional run takes only 5 seconds.
Right.? The way I interpret that is no matter how many more copies of data added, the jit is not getting measurably faster, i.e. it's warmed up.


Second, diz prints out progress information at regular intervals, and it noticeably speeds up.? The jit is certainly kicking in.? Cool.


Third, there are traces listed by jitviewer showing asm code generated.? Again, clearly the jit is doing it's thing.

Perhaps you're implying that the jit is constantly improving the code and will do better with a longer run.? OK, there's no shortage of larger files to try.? When I try the 10MB file dickens from http://www.data-compression.info/Corpora/SilesiaCorpus/index.htm which is over 10x longer and look at the asm code generated (I'm looking at the various traces of output() at line 84) then the code gen remains the "same" as when the shorter frank.txt is used, where same means the parts that seem large still seem large.? I do see a new trace "line 84 run 13165807 times" so the jit is still changing things - I just can't see the improvement.

I think the jit is doing it's thing and more time doesn't change much.? If you want me to try larger files I will.



>I think there is nothing quite broken for PyPy.
OK.? The Pypy asm is working fine, certainly correct, and faster than CPython, but it's still somewhat lengthlier than I was expecting.? Could you explain why a couple parts use much larger alternatives to what I'd expect to see?? 


For line 84 "if (self.low ^ self.high) & 0x80000000 == 0:" I was hopingfor code along the lines of:

(Linux, 64 bit) 


LOAD_ATTR low
LOAD_ATTR high
??? mov    r10,QWORD PTR [rbx+0x8]??? mov    r11,QWORD PTR [rbx+0x16]
BINARY_XOR 
??? mov rax, r10
??? mov rdx, r11
??? xor rax, rdx
BINARY_AND
??? and rax, 0x80000000
COMPARE_OP ==
??? jnz after_if



I see both BINARY_XOR and BINARY_AND call a function instead of xor and and.? Why?? Is there something I can change in my code to let those instructions be used instead?? Can xoring two ints really cause an exception?


BINARY_XOR
??? p120 = call(ConstClass(rbigint.xor), p118, p119, descr=<Callr 8 rr EF=3>)
??? ??? mov QWORD PTR [rbp-0x170],rdx 
??? ??? mov QWORD PTR [rbp-0x178],rax 
??? ??? mov rdi,r15 
??? ??? mov rsi,r13 
??? ??? mov r11d,0x26db2a0 
??? ??? call r11
????guard_no_exception(descr=<Guard239>)
??? ??? cmp QWORD PTR ds:0x457d6e0,0x0 
??? ??? jne 0x3aed3fa2



The load of self.low seems involved as well.? Is there something in the diz code that causes pypy to think it could ever be None/Null?? Is the map lookup for the location of low in self ever condensed to a single instruction?? It seems like the location is calculated using the map at every self.low LOAD_ATTR.? Isn't the point of the map that the "slot" is always the same and could be baked into the load assembly instruction?


so my fantasy?

LOAD_ATTR low
??? mov    r10,QWORD PTR [rbx+0x8]

is really


LOAD_ATTR low
??? p33 = ((pypy.objspace.std.mapdict.W_ObjectObjectSize5)p10).inst_map

??? ??? mov    r10,QWORD PTR [rbx+0x30]???? guard(p33 == ConstPtr(ptr34))

??? ??? movabs r11,0x7fe837ed7be8 

??? ??? cmp    r10,r11 

??? ??? jne    0x3aed3ac1???? p35 = ((pypy.objspace.std.mapdict.W_ObjectObjectSize5)p10).inst__value0

??? ??? mov    r10,QWORD PTR [rbx+0x8]???? guard_nonnull_class(p35, ConstClass(W_LongObject), descr=<Guard213>)

??? ??? cmp    r10,0x1 

??? ??? jb     0x3aed3ad2 

??? ??? cmp    DWORD PTR [r10],0x1308 

??? ??? jne    0x3aed3ad8


>On 32-bit there is extra time needed --- both on PyPy and on CPython --- because the numbers you use overflow signed 32-bit ints, and itneeds longs.

Where are numbers larger than 32 bits ever assigned to self.low or self.high?? I agree that some expressions have temporaries larger than 32 bit, but they're always reduced back to 32 bit before being stored.? If I missed an offending line and fix it to stay within 32 bits, will pypy go with it?? This seems quite important for Windows which only has a 32 bit version.


>an alternate implementation for longs, for example fornumbers that fit into two regular-sized integers.
Too complicated for my needs.? My code can be happy with only 32 bits, I just want Pypy to be happy too.


Thanks Armin,
-Roger

________________________________

From: Armin Rigo <arigo at tunes.org>
To: Roger Flores <aidembb at yahoo.com> 
Cc: "pypy-dev at python.org" <pypy-dev at python.org> 
Sent: Friday, March 1, 2013 2:34 AM
Subject: Re: [pypy-dev] Slow int code

Hi Roger,

On Thu, Feb 28, 2013 at 6:00 PM, Roger Flores <aidembb at yahoo.com> wrote:
> OK then. Unzip it, grab a text file large enough to warm up the jit, and run
> the line to generate the log for jitviewer.

I think there is nothing quite broken for PyPy.? It just has a very
long warm-up time.? On my 64-bit laptop, it runs frank.txt in 14+10
seconds.? If I replace frank.txt by 5 concatenated copies of it, it
takes 34+29 seconds.? That means every additional run takes only 5
seconds.? For comparison the CPython time, also on 64-bit, is 21+21
seconds for frank.txt.? It's not very clear why, but the warm-up time
can be much smaller or bigger depending on the example; it's some
current and future research to improve on that aspect.

On 32-bit there is extra time needed --- both on PyPy and on CPython
--- because the numbers you use overflow signed 32-bit ints, and it
needs longs.? On PyPy we could in theory improve for that case, e.g.
by writing an alternate implementation for longs, for example for
numbers that fit into two regular-sized integers.? (There is actually
one implementation for that, but it's not complete and limited so far,
so not enabled by default; see pypy.objspace.std.smalllongobject.py.
A more complete version would really use two integers, rather than a
"long long" integer.)


A bient?t,

Armin.

From arigo at tunes.org  Fri Mar  1 23:20:13 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 1 Mar 2013 23:20:13 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
Message-ID: <CAMSv6X26aHMvq3dh0YJVSMAsafL7teh9Jz7rcmKs42oUvLEBLg@mail.gmail.com>

Hi Roger,

On Fri, Mar 1, 2013 at 10:13 PM, Roger Flores <aidembb at yahoo.com> wrote:
> I see both BINARY_XOR and BINARY_AND call a function instead of xor and and.  Why?  Is there something I can change in my code to let those instructions be used instead?  Can xoring two ints really cause an exception?

If it becomes calls to, I guess, rbigint.xor and rbigint.and, it's
because these are objects of the Python type "long".  PyPy must
respect the Python semantics: if you do for example "(1 << 80) >> 80",
the result is not 1, but 1L.  This said, I guess that the code is
expected to manipulate 32-bit unsigned integers, and so should not
overflow to long on 64-bit machines.  You may want to investigate, in
the normal Python way, why you get "long" objects there --- this is
not a particularity of PyPy; you can investigate on CPython if you
prefer.

If you're running this code on 32-bit machines, then no chance: you're
getting "long"s.  The only solution would be along the lines I
described before, with a W_TwoIntsLongObject, or even a
W_UnsignedIntLongObject that can store all long objects between 0 and
2**32-1.


A bient?t,

Armin.

From asah at midgard.net  Sat Mar  2 07:29:36 2013
From: asah at midgard.net (Adam Sah)
Date: Fri, 1 Mar 2013 22:29:36 -0800
Subject: [pypy-dev] newbie question: why doesn't pypy run google app engine
	dev_appserver.py?
Message-ID: <CALWCfdJM=1Mc3W9+5A-bEAaC0fDM2hzyCsqxzEtvmQru+HjgRQ@mail.gmail.com>

There are hordes of small app engine apps that would love to self-host but
for the (very) poor performance of dev_appserver, which seems to hinge on
CPU and memory performance, which pypy could address.

thanks,
adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130301/11865ae3/attachment.html>

From fijall at gmail.com  Sat Mar  2 10:41:58 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 2 Mar 2013 11:41:58 +0200
Subject: [pypy-dev] newbie question: why doesn't pypy run google app
 engine dev_appserver.py?
In-Reply-To: <CALWCfdJM=1Mc3W9+5A-bEAaC0fDM2hzyCsqxzEtvmQru+HjgRQ@mail.gmail.com>
References: <CALWCfdJM=1Mc3W9+5A-bEAaC0fDM2hzyCsqxzEtvmQru+HjgRQ@mail.gmail.com>
Message-ID: <CAK5idxR9xjLV+7rBqDJ_LqGNJS+=xzCiH2X62a+sFN8OR2Qv9g@mail.gmail.com>

Hi Adam.

I have no idea. Posting a traceback (or describing the way it does not
work) will be a good start.

On Sat, Mar 2, 2013 at 8:29 AM, Adam Sah <asah at midgard.net> wrote:
> There are hordes of small app engine apps that would love to self-host but
> for the (very) poor performance of dev_appserver, which seems to hinge on
> CPU and memory performance, which pypy could address.
>
> thanks,
> adam
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

From Ronny.Pfannschmidt at gmx.de  Sat Mar  2 10:52:21 2013
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Sat, 02 Mar 2013 10:52:21 +0100
Subject: [pypy-dev] newbie question: why doesn't pypy run google app
 engine dev_appserver.py?
In-Reply-To: <CAK5idxR9xjLV+7rBqDJ_LqGNJS+=xzCiH2X62a+sFN8OR2Qv9g@mail.gmail.com>
References: <CALWCfdJM=1Mc3W9+5A-bEAaC0fDM2hzyCsqxzEtvmQru+HjgRQ@mail.gmail.com>
	<CAK5idxR9xjLV+7rBqDJ_LqGNJS+=xzCiH2X62a+sFN8OR2Qv9g@mail.gmail.com>
Message-ID: <5131CBD5.3080401@gmx.de>

Hi Adam, Maciej

I?m pretty sure there is some vendor lock-in happening.
I vaguely remember app-engine python and its sdk
is a bit different than normal python and
would need some extra porting for support on pypy.

-- Ronny


On 03/02/2013 10:41 AM, Maciej Fijalkowski wrote:
> Hi Adam.
>
> I have no idea. Posting a traceback (or describing the way it does not
> work) will be a good start.
>
> On Sat, Mar 2, 2013 at 8:29 AM, Adam Sah<asah at midgard.net>  wrote:
>> There are hordes of small app engine apps that would love to self-host but
>> for the (very) poor performance of dev_appserver, which seems to hinge on
>> CPU and memory performance, which pypy could address.
>>
>> thanks,
>> adam
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From asah at midgard.net  Sat Mar  2 10:45:44 2013
From: asah at midgard.net (Adam Sah)
Date: Sat, 2 Mar 2013 01:45:44 -0800
Subject: [pypy-dev] newbie question: why doesn't pypy run google app
 engine dev_appserver.py?
In-Reply-To: <CAK5idxR9xjLV+7rBqDJ_LqGNJS+=xzCiH2X62a+sFN8OR2Qv9g@mail.gmail.com>
References: <CALWCfdJM=1Mc3W9+5A-bEAaC0fDM2hzyCsqxzEtvmQru+HjgRQ@mail.gmail.com>
	<CAK5idxR9xjLV+7rBqDJ_LqGNJS+=xzCiH2X62a+sFN8OR2Qv9g@mail.gmail.com>
Message-ID: <CALWCfdJTfB5WmTP9d-T_OHGQimRh0ig6W-XmONqQ6W7H9A7DmQ@mail.gmail.com>

sure... but it looked like nobody's tried it, and I didn't want to dig into
before understanding other people's experience.

the immediate problem was this import error:
https://groups.google.com/d/msg/google-appengine/ijP7X5JXiKM/fIAcI0X-9DIJ

adam


On Sat, Mar 2, 2013 at 1:41 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:

> Hi Adam.
>
> I have no idea. Posting a traceback (or describing the way it does not
> work) will be a good start.
>
> On Sat, Mar 2, 2013 at 8:29 AM, Adam Sah <asah at midgard.net> wrote:
> > There are hordes of small app engine apps that would love to self-host
> but
> > for the (very) poor performance of dev_appserver, which seems to hinge on
> > CPU and memory performance, which pypy could address.
> >
> > thanks,
> > adam
> >
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130302/d71b8e04/attachment.html>

From g2p.code at gmail.com  Sat Mar  2 12:39:55 2013
From: g2p.code at gmail.com (Gabriel de Perthuis)
Date: Sat, 2 Mar 2013 11:39:55 +0000 (UTC)
Subject: [pypy-dev] curses_cffi and others
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
	<kgqh4f$ovd$1@ger.gmane.org>
	<CAMSv6X0n3f+oq-18fqFgq1ntPy=so__UcaxP08A6t15T3Piyyg@mail.gmail.com>
Message-ID: <kgsoeb$l3e$1@ger.gmane.org>

On Fri, 01 Mar 2013 17:44:27 +0100, Armin Rigo wrote:

Le 01 Mar 2013 17:44:27 +0100, Armin Rigo a ?crit:
> Hi Gabriel,
> 
> On Fri, Mar 1, 2013 at 4:22 PM, Gabriel de Perthuis <g2p.code at gmail.com>
> wrote:
>> Packages using CFFI should be able to install-require CFFI whatever the
>> interpreter is, including requiring newer versions than the one the
>> interpreter ships.
>
> (...) 
>
> Even so, more generally, I think that requiring users to install a newer
> version of the interpreter is actually feasible, when we are talking
> about PyPy and not CPython.  A "newer version" of PyPy is just an
> internal improvement, but not a language change --- it's always 2.7.  I
> think that so far, our users are (or should be) ready to deal with this
> minor annoyance; if they are not and really want a completely stable,
> never-changing interpreter, then they're going to stick with CPython
> anyway.
> 
> In other words, I regard this as similar to: "this program requires PyPy
> 2.0 and not a lower version because otherwise its usage of greenlets is
> going to be a huge performance hit".

I don't mind terribly if CFFI is version-locked on PyPy, because PyPy is
currently installed manually by people who know what they are doing, and
they can upgrade it if necessary.

Also, CFFI is great for me because it makes it easy to access just about 
any feature a C library exposes, and does so portably. I can gladly trade 
off CPU speed for improvements to CFFI in featurefulness or usability.

So my previous message was actually a reaction to the "CFFI in stdlib"
inclusion proposal. There are enormous advantages to sharing stdlib
implementation between the interpreters, and using CFFI for stdlib
implementation makes sense. It will make the stdlib better maintained,
and improve both PyPy and CPython.

But speaking as an external user, targetting CPython which is on anything 
but a rolling release, *exposing* the preinstalled version is not 
particularly useful (the bundled version will either be obsolete or not 
widely packaged), and harmful if it interferes at the level of eggs or 
import.

I'll admit this makes me indifferent to expanding the stdlib with anything
but a built-in pip, necessarily-shared resources like a mainloop or data 
types used in API signatures, or de-facto standard libraries that have 
become mature vergeing on stagnant; everything else is better served by a 
blessed subset of PyPI libraries, the way haskell-platform works.

It looks like Stefan Behnel asked the right questions on the more 
appropriate thread (plus some stuff about making compiler invocation 
deterministic that I also care about).



From aidembb at yahoo.com  Sun Mar  3 03:31:06 2013
From: aidembb at yahoo.com (Roger Flores)
Date: Sat, 2 Mar 2013 18:31:06 -0800 (PST)
Subject: [pypy-dev] Slow int code
In-Reply-To: <CAMSv6X26aHMvq3dh0YJVSMAsafL7teh9Jz7rcmKs42oUvLEBLg@mail.gmail.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<CAMSv6X26aHMvq3dh0YJVSMAsafL7teh9Jz7rcmKs42oUvLEBLg@mail.gmail.com>
Message-ID: <1362277866.66877.YahooMailNeo@web162203.mail.bf1.yahoo.com>

Thanks Armin.? This explains a lot.??
I get it better now. 

-Roger


----- Original Message -----
From: Armin Rigo <arigo at tunes.org>
To: Roger Flores <aidembb at yahoo.com>
Cc: "pypy-dev at python.org" <pypy-dev at python.org>
Sent: Friday, March 1, 2013 2:20 PM
Subject: Re: [pypy-dev] Slow int code

Hi Roger,

On Fri, Mar 1, 2013 at 10:13 PM, Roger Flores <aidembb at yahoo.com> wrote:
> I see both BINARY_XOR and BINARY_AND call a function instead of xor and and.? Why?? Is there something I can change in my code to let those instructions be used instead?? Can xoring two ints really cause an exception?

If it becomes calls to, I guess, rbigint.xor and rbigint.and, it's
because these are objects of the Python type "long".? PyPy must
respect the Python semantics: if you do for example "(1 << 80) >> 80",
the result is not 1, but 1L.? This said, I guess that the code is
expected to manipulate 32-bit unsigned integers, and so should not
overflow to long on 64-bit machines.? You may want to investigate, in
the normal Python way, why you get "long" objects there --- this is
not a particularity of PyPy; you can investigate on CPython if you
prefer.

If you're running this code on 32-bit machines, then no chance: you're
getting "long"s.? The only solution would be along the lines I
described before, with a W_TwoIntsLongObject, or even a
W_UnsignedIntLongObject that can store all long objects between 0 and
2**32-1.


A bient?t,

Armin.


From cfbolz at gmx.de  Sun Mar  3 11:21:02 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sun, 03 Mar 2013 11:21:02 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
Message-ID: <5133240E.60308@gmx.de>

On 03/01/2013 10:13 PM, Roger Flores wrote:
>
>
>> I think there is nothing quite broken for PyPy.  It just has a very long warm-up time.
> I think the jit has warmed up, for a few different reasons.  Maybe some of them are leading me astray though.
>
>
>
>> That means every additional run takes only 5 seconds.
> Right.  The way I interpret that is no matter how many more copies of data added, the jit is not getting measurably faster, i.e. it's warmed up.
>
>
> Second, diz prints out progress information at regular intervals, and it noticeably speeds up.  The jit is certainly kicking in.  Cool.
>
>
> Third, there are traces listed by jitviewer showing asm code generated.  Again, clearly the jit is doing it's thing.
>
> Perhaps you're implying that the jit is constantly improving the code and will do better with a longer run.  OK, there's no shortage of larger files to try.  When I try the 10MB file dickens from http://www.data-compression.info/Corpora/SilesiaCorpus/index.htm which is over 10x longer and look at the asm code generated (I'm looking at the various traces of output() at line 84) then the code gen remains the "same" as when the shorter frank.txt is used, where same means the parts that seem large still seem large.  I do see a new trace "line 84 run 13165807 times" so the jit is still changing things - I just can't see the improvement.
>
> I think the jit is doing it's thing and more time doesn't change much.  If you want me to try larger files I will.
>
>
>
>> I think there is nothing quite broken for PyPy.
> OK.  The Pypy asm is working fine, certainly correct, and faster than CPython, but it's still somewhat lengthlier than I was expecting.  Could you explain why a couple parts use much larger alternatives to what I'd expect to see?
>
>
> For line 84 "if (self.low ^ self.high) & 0x80000000 == 0:" I was hopingfor code along the lines of:
>
> (Linux, 64 bit)
>
>
> LOAD_ATTR low
> LOAD_ATTR high
>      mov    r10,QWORD PTR [rbx+0x8]    mov    r11,QWORD PTR [rbx+0x16]
> BINARY_XOR
>      mov rax, r10
>      mov rdx, r11
>      xor rax, rdx
> BINARY_AND
>      and rax, 0x80000000
> COMPARE_OP ==
>      jnz after_if
>
>
>
> I see both BINARY_XOR and BINARY_AND call a function instead of xor and and.  Why?  Is there something I can change in my code to let those instructions be used instead?  Can xoring two ints really cause an exception?
>
>
> BINARY_XOR
>      p120 = call(ConstClass(rbigint.xor), p118, p119, descr=<Callr 8 rr EF=3>)
>          mov QWORD PTR [rbp-0x170],rdx
>          mov QWORD PTR [rbp-0x178],rax
>          mov rdi,r15
>          mov rsi,r13
>          mov r11d,0x26db2a0
>          call r11
>      guard_no_exception(descr=<Guard239>)
>          cmp QWORD PTR ds:0x457d6e0,0x0
>          jne 0x3aed3fa2

Are you *sure* you are running on a 64 bit machine? When I run diz.py
on a 64 bit machine, the BINARY_XOR bytecodes turn into int_xor low
level operations, as expected. On 32 bit I get the call, as you pasted.

Anyway, to debug where low and high turn into Python longs, you can put
the following properties in arithmetic32.Encoder:

     def get_low(self):
         return self._low

     def set_low(self, val):
         assert isinstance(val, int)
         self._low = val

     low = property(get_low, set_low)

     def get_high(self):
         return self._high

     def set_high(self, val):
         assert isinstance(val, int)
         self._high = val

     high = property(get_high, set_high)

Indeed, they didn't trigger on 64 bit for me. In 32 bit they obviously
trigger because of the line self.low = (self.low << 1) & 0xffffffff
(0xffffffff is a Python long on 32 bit).

Cheers,

Carl Friedrich

From aidembb at yahoo.com  Mon Mar  4 09:42:43 2013
From: aidembb at yahoo.com (Roger Flores)
Date: Mon, 4 Mar 2013 00:42:43 -0800 (PST)
Subject: [pypy-dev] Slow int code
In-Reply-To: <513323F7.40706@gmx.de>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
Message-ID: <1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>

On March 3, 2013 2:20 AM, Carl Friedrich Bolz wrote:
>Are you *sure* you are running on a 64 bit machine?

Sure?? No.? I assumed it's 64bit pypy because it was generating x86_64 instructions.? How would you check for sure?

uname reports x86_64 on the machine I built pypy on.

$ pypy --version
Python 2.7.3 (42c0d1650cf4, Feb 23 2013, 01:53:42)
[PyPy 2.0.0-beta1 with GCC 4.6.3]

That doesn't show the machine size.

pypy --info is interesting but doesn't help either

?


>When I run diz.py on a 64 bit machine, the BINARY_XOR bytecodes turn into int_xor low
level operations, as expected.

I would like to see what you see using jitviewer since it differs from what I pasted (and maybe to figure out why).? 



>Anyway, to debug where low and high turn into Python longs, you can putthe following properties in arithmetic32.Encoder:
...

That was some clever code. I like it. :)


> In 32 bit they obviously trigger because of the line self.low = (self.low << 1) & 0xffffffff
(0xffffffff is a Python long on 32 bit).

Specifically it's a long because 0xffffffff is unsigned 32 bit and thus can't fit in a signed 32bit.? There no hope for the 32 bit version unless more int types are added beyond Python's two, specifically an uint32 type.? And that's not python.? Except, I did notice that numpy has this type and more:

https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/npy_common.h

It's conceivable that the efforts to bring numpy into pypy will be dealing with these various int sizes and could better support uint32 code like that in diz.? However, I suspect it's not a huge win because the int operations will remain function calls instead of single x86 instructions.? That's the real pain, on all platforms.


Anyways, I run the code and it works.? Everywhere.? And I've finally convinced myself to stop abusing the dict with millions of items, so I've got more stuff to do. :)



-Roger

From fijall at gmail.com  Mon Mar  4 10:12:53 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 4 Mar 2013 11:12:53 +0200
Subject: [pypy-dev] Slow int code
In-Reply-To: <1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
Message-ID: <CAK5idxResAZTkzrhaErE4eH68H678pccdgC-pmsbihe=Oiyd6g@mail.gmail.com>

import sys
print sys.maxint

On Mon, Mar 4, 2013 at 10:42 AM, Roger Flores <aidembb at yahoo.com> wrote:
> On March 3, 2013 2:20 AM, Carl Friedrich Bolz wrote:
>>Are you *sure* you are running on a 64 bit machine?
>
> Sure?  No.  I assumed it's 64bit pypy because it was generating x86_64 instructions.  How would you check for sure?
>
> uname reports x86_64 on the machine I built pypy on.
>
> $ pypy --version
> Python 2.7.3 (42c0d1650cf4, Feb 23 2013, 01:53:42)
> [PyPy 2.0.0-beta1 with GCC 4.6.3]
>
> That doesn't show the machine size.
>
> pypy --info is interesting but doesn't help either
>
> ?
>
>
>>When I run diz.py on a 64 bit machine, the BINARY_XOR bytecodes turn into int_xor low
> level operations, as expected.
>
> I would like to see what you see using jitviewer since it differs from what I pasted (and maybe to figure out why).
>
>
>
>>Anyway, to debug where low and high turn into Python longs, you can putthe following properties in arithmetic32.Encoder:
> ...
>
> That was some clever code. I like it. :)
>
>
>> In 32 bit they obviously trigger because of the line self.low = (self.low << 1) & 0xffffffff
> (0xffffffff is a Python long on 32 bit).
>
> Specifically it's a long because 0xffffffff is unsigned 32 bit and thus can't fit in a signed 32bit.  There no hope for the 32 bit version unless more int types are added beyond Python's two, specifically an uint32 type.  And that's not python.  Except, I did notice that numpy has this type and more:
>
> https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/npy_common.h
>
> It's conceivable that the efforts to bring numpy into pypy will be dealing with these various int sizes and could better support uint32 code like that in diz.  However, I suspect it's not a huge win because the int operations will remain function calls instead of single x86 instructions.  That's the real pain, on all platforms.
>
>
> Anyways, I run the code and it works.  Everywhere.  And I've finally convinced myself to stop abusing the dict with millions of items, so I've got more stuff to do. :)
>
>
>
> -Roger
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From anto.cuni at gmail.com  Mon Mar  4 10:20:34 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 04 Mar 2013 10:20:34 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
Message-ID: <51346762.3050907@gmail.com>

On 03/04/2013 09:42 AM, Roger Flores wrote:
> On March 3, 2013 2:20 AM, Carl Friedrich Bolz wrote:
>> >Are you*sure*  you are running on a 64 bit machine?
> Sure?  No.  I assumed it's 64bit pypy because it was generating x86_64 instructions.  How would you check for sure?
>
> uname reports x86_64 on the machine I built pypy on.
>
> $ pypy --version
> Python 2.7.3 (42c0d1650cf4, Feb 23 2013, 01:53:42)
> [PyPy 2.0.0-beta1 with GCC 4.6.3]
>
> That doesn't show the machine size.
>
> pypy --info is interesting but doesn't help either

just a wild guess: is it possible that you generated pyc files with a 32bit 
version of pypy and then imported it on a 64bit one?

For example, suppose you have this foo.py:

def foo():
     return 2147483648
print type(foo())

if you import it on 32bit, it prints 'long' and generates a pyc file. If you 
then import 'foo' on 64bit, it still prints 'long', but if you remove the pyc 
and import again, it prints 'int'.
(This happens because 2147483648 is stored as a long inside the marshalled pyc 
file).

ciao,
Anto

From arigo at tunes.org  Mon Mar  4 15:26:32 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Mar 2013 15:26:32 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <51346762.3050907@gmail.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<51346762.3050907@gmail.com>
Message-ID: <CAMSv6X1dAmqHTUBqADDYhJXcXatw6DnvMw07Qs7ppgFDUrPjfQ@mail.gmail.com>

Hi Anto,

On Mon, Mar 4, 2013 at 10:20 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> (This happens because 2147483648 is stored as a long inside the marshalled
> pyc file).

Riiiight, absolutely.  How about deciding that this is a bug and
fixing it in PyPy (even if it's the CPython behavior)?  It can even be
done while keeping the same .pyc file, if we make a simple extension
to the .pyc file format for "integers without an L suffix that fit in
64-bit".  (The .pyc files are anyway different from CPython's.
Moreover we need to propagate this informtion from the compiler, which
means it would only be used when marshalling code objects, never
directly integers --- important for compatibility with CPython, when
"marshal" is used directly.)


A bient?t,

Armin.

From bokr at oz.net  Mon Mar  4 16:32:37 2013
From: bokr at oz.net (Bengt Richter)
Date: Mon, 04 Mar 2013 16:32:37 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <51346762.3050907@gmail.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>	<513323F7.40706@gmx.de>	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<51346762.3050907@gmail.com>
Message-ID: <kh2eum$hns$1@ger.gmane.org>

On 03/04/2013 10:20 AM Antonio Cuni wrote:
> On 03/04/2013 09:42 AM, Roger Flores wrote:
>> On March 3, 2013 2:20 AM, Carl Friedrich Bolz wrote:
>>>> Are you*sure*  you are running on a 64 bit machine?
>> Sure?  No.  I assumed it's 64bit pypy because it was generating x86_64 instructions.  How would you check for sure?
>>
>> uname reports x86_64 on the machine I built pypy on.
>>
>> $ pypy --version
>> Python 2.7.3 (42c0d1650cf4, Feb 23 2013, 01:53:42)
>> [PyPy 2.0.0-beta1 with GCC 4.6.3]
>>
>> That doesn't show the machine size.
>>
>> pypy --info is interesting but doesn't help either
>
> just a wild guess: is it possible that you generated pyc files with a 32bit
> version of pypy and then imported it on a 64bit one?
>
> For example, suppose you have this foo.py:
>
> def foo():
>       return 2147483648
> print type(foo())
>
> if you import it on 32bit, it prints 'long' and generates a pyc file. If you
> then import 'foo' on 64bit, it still prints 'long', but if you remove the pyc
> and import again, it prints 'int'.
> (This happens because 2147483648 is stored as a long inside the marshalled pyc
> file).
>
ISTM like such a cross-import should print a warning or something (maybe silent
reimport?) even if you don't want to call it a bug.

<rant_warning>
Something bothers me about calling alternate representations different "types."
ISTM HLL types are (or should be) abstract, and concrete representation choices
are implementation details.

I'm ok with "type" from the vocabulary of C, where it *does* (now that C99
supplies specific representation-sized typedefs without having to hack them
ourselves ;-) map to choices of representation of different abstract types.

But -- I am not comfortable with type(2**31) printing two different things
in the same *HL* language, unless it is an introspection into the implementation,
in which case I think it ought not to be called "type" but instead maybe for example
"rtype" (for representation-type),and let "type" be reserved for abstract types
of the language. Thus we might get, e.g., (faked OTTOMH):
     >>> type(2**31)  # on either 64 or 32 bit interpreter
     <type 'int'>
     >>> type(2**63)  # on either 64 or 32 bit interpreter
     <type 'int'>
and current behavior extended via "rtype"
     >>> rtype(2**31)  # on 64 bit interpreter
     <rtype 'int64_t'>
     >>> rtype(2**31)  # on 32 bit interpreter
     <rtype 'intBInnn'>	# BI for BigInt of some design nnn
     >>> rtype(2**30)  # on 32 bit interpreter
     <rtype 'int32_t'>
     >>> rtype(2**63)  # on either 64 or 32 bit interpreter
     <rtype 'intBInnn'>	# BI for BigInt, nnn to identify different versions if desired
by the same token, arguably an abstract string is a sequence of abstract characters,and
     >>> type(''), type(u'') 	# should give you
     (<type 'string'>, <type 'string'>)	# not (<type 'str'>, <type 'unicode'>)
whereas rtype could reveal strings as being represented as byte sequences with or latin1 or(
dingbat encoding, or utf8 or various other unicode or wchar_t unicode etc.

Might be interesting if rtype could track jitted and ffi representations too, and still
have type say the right thing for type(thing). Sorry, got carried away ;-/
</rant_warning>

BTW, what happens if you import in the opposite order?
Can the 32-bit interpreter process the 64-bit pyc?

Regards,
Bengt Richter


From arigo at tunes.org  Mon Mar  4 16:52:01 2013
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Mar 2013 16:52:01 +0100
Subject: [pypy-dev] Slow int code
In-Reply-To: <kh2eum$hns$1@ger.gmane.org>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<51346762.3050907@gmail.com> <kh2eum$hns$1@ger.gmane.org>
Message-ID: <CAMSv6X32ryXm2gmc+EnM5WQq5tGkwkq_8J=DQAkuOeCfLFX2tw@mail.gmail.com>

Hi Bengt,

On Mon, Mar 4, 2013 at 4:32 PM, Bengt Richter <bokr at oz.net> wrote:
> </rant_warning>

That's an issue of the Python language, which we won't address here.
Note also that Python 3 goes a bit in the direction you describe.

> BTW, what happens if you import in the opposite order?
> Can the 32-bit interpreter process the 64-bit pyc?

Yes: in this case the 64-bit Python (either CPython or PyPy) will
store the 64-bit "int", and the 32-bit Python will go ``oups, a 64-bit
int, I'm going to load it as a "long"''.

Actually the only thing we could change (assuming we want to fix it)
would be, on 32-bit, to also generate such a 64-bit int for code
objects that contain 64-bit Python constants without the "L" suffix.


A bient?t,

Armin.

From aidembb at yahoo.com  Mon Mar  4 22:38:45 2013
From: aidembb at yahoo.com (Roger Flores)
Date: Mon, 4 Mar 2013 13:38:45 -0800 (PST)
Subject: [pypy-dev] Slow int code
In-Reply-To: <51346762.3050907@gmail.com>
References: <1361995804.94140.YahooMailNeo@web162205.mail.bf1.yahoo.com>
	<CAFRnB2VZnjy=+WNAipZHkgOtvfPsV6O8gzjrf6NG+i6e4Ae5EQ@mail.gmail.com>
	<1361997160.20274.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAFRnB2W-1-ehf+5ShcEZtoPxj5srLVUfszRrKPRA72RjiMgTjg@mail.gmail.com>
	<1361998548.58023.YahooMailNeo@web162202.mail.bf1.yahoo.com>
	<CAMSv6X1fkkjzzNBak64o7o7WKpRABUY9v3ZmUUtFC74He6A0aA@mail.gmail.com>
	<1362070815.93192.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<CAMSv6X3wUk+y67Zj51RpdV6VQVZZk5QmbH9kw+K1DW2THSF+mw@mail.gmail.com>
	<1362172403.92948.YahooMailNeo@web162204.mail.bf1.yahoo.com>
	<513323F7.40706@gmx.de>
	<1362386563.13550.YahooMailNeo@web162206.mail.bf1.yahoo.com>
	<51346762.3050907@gmail.com>
Message-ID: <1362433125.74792.YahooMailNeo@web162205.mail.bf1.yahoo.com>



On Mon, Mar 4, 2013 at 1:13 AM, Maciej Fijalkowski wrote:
>print sys.maxint
>>>> print hex(sys.maxint)
0x7fffffffffffffff

That works.? Not so obvious though.



On Monday, March 4, 2013 1:20 AM, Antonio Cuni wrote:
>just a wild guess: is it possible that you generated pyc files with a 32bit version of pypy and then imported it on a 64bit one?

Ding!? That's exactly what was afflicting me.? I had no idea the pyc files would not get refreshed on the platform change.


After 'rm *pyc' I now see substantially better code!? Importantly, the arithmetic function calls are reduced to the single x86 instructions that Carl Friedrich saw.


For the record, this is what I now see:

BINARY_XOR 
??? i39 = ((pypy.objspace.std.intobject.W_IntObject)p35).inst_intval [pure]
??? ??? mov? ? r8,QWORD PTR [r10+0x8]???? i40 = ((pypy.objspace.std.intobject.W_IntObject)p37).inst_intval [pure]
??? ??? mov? ? rdi,QWORD PTR [r13+0x8]???? i41 = int_xor(i39, i40)
??? ??? mov? ? rcx,r8 
??? ??? xor? ? r8,rdiLOAD_CONST 2147483648
BINARY_AND 
???? i43 = i41 & 2147483648
??? ??? mov? ? r11d,0x80000000 
??? ??? and? ? r8,r11LOAD_CONST 0
COMPARE_OP ==
???? i45 = i43 == 0
??? ??? cmp? ? r8,0x0 
??? ??? jne? ? 0x9468582b

It's strange the AND doesn't use the imm32 form like the COMPARE does.? It's unfortunate the COMPARE doesn't suppress the cmp and reuse the zero flag from the prior instruction.

Looking at the self.low and self.high references prior to the XOR, I'm assuming the null guard is 
because their types are a PyInt_Type which can always have a None (Python semantics).? But I don't understand why the work isn't moved out of the loop (or maybe it is and I just can't tell).



Thanks everyone,
-Roger


From samuele.pedroni at gmail.com  Tue Mar  5 21:05:12 2013
From: samuele.pedroni at gmail.com (Samuele Pedroni)
Date: Tue, 5 Mar 2013 21:05:12 +0100
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
Message-ID: <CAGwLUHYMsW6G15ejdput-qz_ZuTXUrFjK0YqjTi4mL774xwC2w@mail.gmail.com>

part of what cpython does (did?) in this sort of cases is to ship a
*.egg-info for the package that has an external existence as well, like in
CPython2.7 you can see there are .egg-info files for wsgiref and argparse
in the stdlib dir.


On Fri, Mar 1, 2013 at 11:54 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all, hi Jeremy,
>
> I'm thinking about merging the curses_cffi branch, and I see efforts
> going on in sqlite-cffi.  Great :-)
>
> We need to think about how we want to support cffi-based modules in
> PyPy.  So far the idea is to include the official release of CFFI
> together with regular PyPy installations.  Then, how can we disable
> "pip install cffi" from breaking everything up by installing a
> different release of cffi on top of PyPy?  Is it enough to detect in
> cffi's setup.py "oh, it's PyPy and cffi is already installed, so I'll
> do nothing"?  Should it still check that the version number matches,
> and complain if it doesn't?
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130305/79f86d9c/attachment.html>

From naomi at nthmost.com  Thu Mar  7 22:59:25 2013
From: naomi at nthmost.com (Naomi Most)
Date: Thu, 7 Mar 2013 13:59:25 -0800
Subject: [pypy-dev] Dinner w/ Noisebridge hackers next week?
Message-ID: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>

Hi guys,

Responding to the blog post about a San Francisco visit -- a handful
of us Python oriented hackers at Noisebridge would love to bring you
to the space and take you to dinner.

I propose Monday evening to meet at Noisebridge around 7pm.  Tuesday
evening at 6pm would also work for most of us.

Let me know if you can do it, and we'll put the announcement up on the
Noisebridge list asap.  Any time you want to spend would be fantastic,
whether you want to talk at us or do a discussion at NB for a while
and then get dinner, or just have dinner, or novel combinations such
as cooking dinner at Noisebridge (not really recommended).

Thanks -- hope to meet you soon!

Cheers,
Naomi

-- 
Naomi Theora Most
naomi at nthmost.com
+1-415-728-7490

skype: nthmost

http://twitter.com/nthmost

From mummylauncher at gmail.com  Thu Mar  7 17:42:26 2013
From: mummylauncher at gmail.com (Charles Roland)
Date: Thu, 7 Mar 2013 11:42:26 -0500
Subject: [pypy-dev] Translating --sandbox -02 python interpreter for windows
Message-ID: <CAAAcipit+8R0Q5invN6w=0jiB++u+ieMWc-tb_JXhark=_PPHw@mail.gmail.com>

Hello friends at pypy-dev,

   My goal is to get the sandbox python interpreter working in windows.  I
was able to get a python-c.exe built using the --sandbox and -O2 options
with pypy and translate.py.  The next step was to test it out and that is
where my problem is.  I ran this script:
(pypysource)\pypy\translator\sandbox\interact.py
with the python-c.exe I built as an argument.  I tried running that script
in pypy and Cpython2.7 and got this error with both interpreters:

  File "c:\pypy-2.0-beta1\lib-python\2.7\subprocess.py", line 637, in
__init__
    raise ValueError("close_fds is not supported on Windows "
ValueError: close_fds is not supported on Windows platforms if you redirect
stdin/stdout/stderr

>From what I know about the sandbox system, there IS stdin/out redirection
going on somewhere in the mix (sandbox interpreter is being redirected to
controller script).  Is this the redirection the warning is talking about?
 Is this fixable in a simple way?

Thank you all for developing pypy and taking the time to read my question.

- Charles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130307/89e51993/attachment.html>

From fijall at gmail.com  Thu Mar  7 23:25:00 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Mar 2013 00:25:00 +0200
Subject: [pypy-dev] Dinner w/ Noisebridge hackers next week?
In-Reply-To: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>
References: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>
Message-ID: <CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>

On Thu, Mar 7, 2013 at 11:59 PM, Naomi Most <naomi at nthmost.com> wrote:
> Hi guys,
>
> Responding to the blog post about a San Francisco visit -- a handful
> of us Python oriented hackers at Noisebridge would love to bring you
> to the space and take you to dinner.
>
> I propose Monday evening to meet at Noisebridge around 7pm.  Tuesday
> evening at 6pm would also work for most of us.
>
> Let me know if you can do it, and we'll put the announcement up on the
> Noisebridge list asap.  Any time you want to spend would be fantastic,
> whether you want to talk at us or do a discussion at NB for a while
> and then get dinner, or just have dinner, or novel combinations such
> as cooking dinner at Noisebridge (not really recommended).
>
> Thanks -- hope to meet you soon!
>
> Cheers,
> Naomi

Hi Naomi.

I'm only landing on Tuesday afternoon (if we're talking about this
Tuesday), so Monday is off since I'm in transit. I'm fine with Tuesday
though, provided everyone else is ready to dine after the whole
travel.

>
> --
> Naomi Theora Most
> naomi at nthmost.com
> +1-415-728-7490
>
> skype: nthmost
>
> http://twitter.com/nthmost
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From fijall at gmail.com  Thu Mar  7 23:38:31 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Mar 2013 00:38:31 +0200
Subject: [pypy-dev] Dinner w/ Noisebridge hackers next week?
In-Reply-To: <CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>
References: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>
	<CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>
Message-ID: <CAK5idxTFQqV=K_-GtyU9WKf=_bqbmosF8uVGkR3sYkkxSy0ogA@mail.gmail.com>

On Fri, Mar 8, 2013 at 12:25 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Thu, Mar 7, 2013 at 11:59 PM, Naomi Most <naomi at nthmost.com> wrote:
>> Hi guys,
>>
>> Responding to the blog post about a San Francisco visit -- a handful
>> of us Python oriented hackers at Noisebridge would love to bring you
>> to the space and take you to dinner.
>>
>> I propose Monday evening to meet at Noisebridge around 7pm.  Tuesday
>> evening at 6pm would also work for most of us.
>>
>> Let me know if you can do it, and we'll put the announcement up on the
>> Noisebridge list asap.  Any time you want to spend would be fantastic,
>> whether you want to talk at us or do a discussion at NB for a while
>> and then get dinner, or just have dinner, or novel combinations such
>> as cooking dinner at Noisebridge (not really recommended).
>>
>> Thanks -- hope to meet you soon!
>>
>> Cheers,
>> Naomi
>
> Hi Naomi.
>
> I'm only landing on Tuesday afternoon (if we're talking about this
> Tuesday), so Monday is off since I'm in transit. I'm fine with Tuesday
> though, provided everyone else is ready to dine after the whole
> travel.
>
>>
>> --
>> Naomi Theora Most
>> naomi at nthmost.com
>> +1-415-728-7490
>>
>> skype: nthmost
>>
>> http://twitter.com/nthmost
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev

Hi Again.

Actually Tuesday 6pm sounds perfect for us (if you can allow us to be
a little jetlagged) :-)

Cheers,
fijal

From arigo at tunes.org  Thu Mar  7 23:38:48 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 7 Mar 2013 23:38:48 +0100
Subject: [pypy-dev] Dinner w/ Noisebridge hackers next week?
In-Reply-To: <CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>
References: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>
	<CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>
Message-ID: <CAMSv6X2DDOphMU+Bz=+cf=-woSbVx0rfctGCgVDeL7g4HMdbag@mail.gmail.com>

Hi Naomi,

On Thu, Mar 7, 2013 at 11:25 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> I'm only landing on Tuesday afternoon (if we're talking about this
> Tuesday), so Monday is off since I'm in transit. I'm fine with Tuesday
> though, provided everyone else is ready to dine after the whole
> travel.

Yes, that's fine with me as well.  I'm arriving Monday evening but I
wouldn't be there for 7pm anyway.  Tuesday evening matches perfectly
my plans.


A bient?t,

Armin.

From naomi at nthmost.com  Thu Mar  7 23:55:07 2013
From: naomi at nthmost.com (Naomi Most)
Date: Thu, 7 Mar 2013 14:55:07 -0800
Subject: [pypy-dev] Dinner w/ Noisebridge hackers next week?
In-Reply-To: <CAMSv6X2DDOphMU+Bz=+cf=-woSbVx0rfctGCgVDeL7g4HMdbag@mail.gmail.com>
References: <CAPKWcoxZaPf7CJSpjdxARmyAeP77V7wYksicMG54x8t6cvp0tA@mail.gmail.com>
	<CAK5idxTQ21ZRcWY6Z-=u49X9+fzZuKbJDd641ZY_hfUAdO3sWA@mail.gmail.com>
	<CAMSv6X2DDOphMU+Bz=+cf=-woSbVx0rfctGCgVDeL7g4HMdbag@mail.gmail.com>
Message-ID: <CAPKWcowHONsu7g-7Z4Dhu5C2mhWK36_=Ju2KEaCS_-pUk2UizQ@mail.gmail.com>

Sweet!  Yes, this coming Tuesday.  Didn't realize Monday would be
in-transit.  I hope Tuesday evening you'll all be feeling relatively
recovered.

Come to Noisebridge at 6pm on Tuesday, we'll grab dinner nearby.  And
it's Noisebridge, so all are welcome.

Noisebridge
2169 Mission St
San Francisco, CA

Cheers,
Naomi




On Thu, Mar 7, 2013 at 2:38 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Naomi,
>
> On Thu, Mar 7, 2013 at 11:25 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> I'm only landing on Tuesday afternoon (if we're talking about this
>> Tuesday), so Monday is off since I'm in transit. I'm fine with Tuesday
>> though, provided everyone else is ready to dine after the whole
>> travel.
>
> Yes, that's fine with me as well.  I'm arriving Monday evening but I
> wouldn't be there for 7pm anyway.  Tuesday evening matches perfectly
> my plans.
>
>
> A bient?t,
>
> Armin.



-- 
Naomi Theora Most
naomi at nthmost.com
+1-415-728-7490

skype: nthmost

http://twitter.com/nthmost

From jameslan at gmail.com  Fri Mar  8 02:09:09 2013
From: jameslan at gmail.com (James Lan)
Date: Thu, 7 Mar 2013 17:09:09 -0800
Subject: [pypy-dev] Fwd: test_all.py can't recognize has multi inherited
	classes
In-Reply-To: <CAOh=ABzX+OcCUnejVfNnsmB9UgBiG6-hpm67RyECP1WNAu6Lsw@mail.gmail.com>
References: <CAOh=ABzoDH8mtFDpA6Ge+nFY0Aqy7tupkHgHsQ+msB8O4_PJrg@mail.gmail.com>
	<511DE501.4080707@gmx.de> <511DE7A8.7090803@gmx.de>
	<CAK5idxTr5Hpe41WGriLjq98HWTrFit5-gvPWoHP7Aw+EuamziA@mail.gmail.com>
	<511DEB73.2040602@gmx.de>
	<CAOh=ABzX+OcCUnejVfNnsmB9UgBiG6-hpm67RyECP1WNAu6Lsw@mail.gmail.com>
Message-ID: <CAOh=ABw-GNwKB+=4uv6HU8B3rTah_xq-gJTQsnz-5ZoAQkVKLw@mail.gmail.com>

forgot to send to mailing list
---------- Forwarded message ----------
From: James Lan <jameslan at gmail.com>
Date: Wed, Mar 6, 2013 at 2:39 PM
Subject: Re: [pypy-dev] test_all.py can't recognize has multi inherited
classes
To: Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>


Hi,

Is there any update on this?

Do I need to file a bug?

Thanks,
James


On Fri, Feb 15, 2013 at 12:01 AM, Ronny Pfannschmidt <
Ronny.Pfannschmidt at gmx.de> wrote:

> On 02/15/2013 08:55 AM, Maciej Fijalkowski wrote:
>
>> On Fri, Feb 15, 2013 at 9:45 AM, Ronny Pfannschmidt
>> <Ronny.Pfannschmidt at gmx.de>  wrote:
>>
>>> i confirmed the issue - its in the included pytest and its fixed upstream
>>>
>>> its going to be fixed in the pytest branch of pypy later
>>>
>>
>> Can't you fix it on default?
>>
>
> i need to investigate the effects of the update first
>
>
>
>>
>>>
>>>
>>>
>>> On 02/15/2013 08:34 AM, Ronny Pfannschmidt wrote:
>>>
>>>>
>>>> Hi James,
>>>>
>>>>
>>>> Interesting find,
>>>>
>>>> i can replicate and i don't yet see how it happens,
>>>> because the code pytest uses does walk the mro to find tests in parent
>>>> classes
>>>>
>>>> i will report back once i figured whats wrong
>>>>
>>>> -- Ronny
>>>>
>>>>
>>>> On 02/15/2013 08:14 AM, James Lan wrote:
>>>>
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I tried to run test cases for jvm backend by the following command,
>>>>>
>>>>>
>>>>> pypy pypy/test_all.py rpython/translator/jvm/test
>>>>>
>>>>>
>>>>> but what I got was 10 tests defined in
>>>>> rpython/translator/jvm/test/**test_backendopt.py although there are
>>>>> two
>>>>> dozens of test files in that directory.
>>>>>
>>>>> After experiment, I found that it failed to collect test within a multi
>>>>> inherited class. Take test_constant.py as an example, it defines a
>>>>> class,
>>>>>
>>>>>
>>>>> class TestConstant(BaseTestConstant, JvmTest):
>>>>> pass
>>>>>
>>>>>
>>>>> and JvmTest is defined as,
>>>>>
>>>>>
>>>>> class JvmTest(BaseRtypingTest, OORtypeMixin):
>>>>> ......
>>>>>
>>>>>
>>>>> When I executed the folloing,
>>>>>
>>>>>
>>>>> ypy pypy/test_all.py rpython/translator/jvm/test/**test_constant.py
>>>>> --collectonly
>>>>>
>>>>>
>>>>> it reported 0 item was collected. If I remove BaseTestConstant from
>>>>> TestConstant's super classes, it still reports 0 item collected; but if
>>>>> I remove JvmTest from its super classes, it reports 18 items are
>>>>> collected.
>>>>>
>>>>>
>>>>> I'm wondering how the daily test solve this problem?
>>>>>
>>>>> Thanks,
>>>>> James Lan
>>>>>
>>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130307/f69bdd38/attachment.html>

From fijall at gmail.com  Fri Mar  8 11:32:28 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Mar 2013 12:32:28 +0200
Subject: [pypy-dev] instruction cache and data cache on ARM
Message-ID: <CAK5idxTTi2uqNJU_81_Z2c0fazPee7tAYBqeUMJMdMQNtvijyA@mail.gmail.com>

Looking at random stuff I found this:
http://luajit.org/download/v2.0.1_hotfix1.patch which means ARM might
have separate instruction cache and data cache that don't sync to each
other. Can someone confirm/deny that we have to flush them?

Cheers,
fijal

From david.schneider at picle.org  Fri Mar  8 12:01:44 2013
From: david.schneider at picle.org (David Schneider)
Date: Fri, 8 Mar 2013 12:01:44 +0100
Subject: [pypy-dev] instruction cache and data cache on ARM
In-Reply-To: <CAK5idxTTi2uqNJU_81_Z2c0fazPee7tAYBqeUMJMdMQNtvijyA@mail.gmail.com>
References: <CAK5idxTTi2uqNJU_81_Z2c0fazPee7tAYBqeUMJMdMQNtvijyA@mail.gmail.com>
Message-ID: <E813BC9D-79AC-400A-8A4D-4811B111845B@picle.org>

Hi,

That is correct, we currently flush at least the instruction cache when we emit machine code to memory.

David

On 08.03.2013, at 11:32, Maciej Fijalkowski <fijall at gmail.com> wrote:

> Looking at random stuff I found this:
> http://luajit.org/download/v2.0.1_hotfix1.patch which means ARM might
> have separate instruction cache and data cache that don't sync to each
> other. Can someone confirm/deny that we have to flush them?
> 
> Cheers,
> fijal
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From gw.2013 at tnode.com  Fri Mar  8 20:08:31 2013
From: gw.2013 at tnode.com (GW)
Date: Fri, 08 Mar 2013 20:08:31 +0100
Subject: [pypy-dev] =?utf-8?q?Integrate_Clyther_into_PyPy=3F?=
Message-ID: <5f6036301f1a943a6b2ea059433579e5@tnode.com>

  

Hello, 

I am wondering why is there a proposal to provide only a
compatible working NumPy CPU implementation for PyPy? Aren't NumPy
operations more suited for GPGPU (eg. OpenCL or CUDA) and supporting
them would result in a greater performance boost than just CPU? 

A
great project thet seems to have implemented all aspects of
parallization, also uses JIT ideas, and has a NumPy-compatible interface
(with exchangable backend/context implementation) is Clyther
(http://srossross.github.com/Clyther/ [1]). In my opinion it would be
great to join the efforts, combine the knowledge and code, and create
something really powerful. 

Maybe the developers, knowledge, or code
from Copperhead (http://copperhead.github.com/ [2]) could also help
(although it seems they are approaching the problem differently).


Greetings,
 gw 

Links:
------
[1]
http://srossross.github.com/Clyther/
[2] http://copperhead.github.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130308/225dfad5/attachment.html>

From arigo at tunes.org  Fri Mar  8 22:24:00 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 8 Mar 2013 22:24:00 +0100
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAGwLUHYMsW6G15ejdput-qz_ZuTXUrFjK0YqjTi4mL774xwC2w@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
	<CAGwLUHYMsW6G15ejdput-qz_ZuTXUrFjK0YqjTi4mL774xwC2w@mail.gmail.com>
Message-ID: <CAMSv6X2jVX=_8Fr80X0RR6mhCRNXpm8YktqEMBe=qUeP71o8hw@mail.gmail.com>

Hi Samuele,

On Tue, Mar 5, 2013 at 9:05 PM, Samuele Pedroni
<samuele.pedroni at gmail.com> wrote:
> part of what cpython does (did?) in this sort of cases is to ship a
> *.egg-info for the package that has an external existence as well, like in
> CPython2.7 you can see there are .egg-info files for wsgiref and argparse in
> the stdlib dir.

Thanks for the tip.  The presence of the .egg-info file is enough to
convince both pip and easy_install that the package is already
installed?


A bient?t,

Armin.

From samuele.pedroni at gmail.com  Fri Mar  8 22:45:46 2013
From: samuele.pedroni at gmail.com (Samuele Pedroni)
Date: Fri, 8 Mar 2013 22:45:46 +0100
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAMSv6X2jVX=_8Fr80X0RR6mhCRNXpm8YktqEMBe=qUeP71o8hw@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
	<CAGwLUHYMsW6G15ejdput-qz_ZuTXUrFjK0YqjTi4mL774xwC2w@mail.gmail.com>
	<CAMSv6X2jVX=_8Fr80X0RR6mhCRNXpm8YktqEMBe=qUeP71o8hw@mail.gmail.com>
Message-ID: <CAGwLUHaN6B5jMTp9NDO3LRp6oRdkAKQDOOL24NHZ38OG-J+GOg@mail.gmail.com>

hi, think so, here is what I get if I try to install the versions already
there with their egg-info files:

$ pip install argparse==1.2.1
Requirement already satisfied (use --upgrade to upgrade): argparse==1.2.1
in /usr/lib/python2.7
Cleaning up...
$ pip install wsgiref==0.1.2
Requirement already satisfied (use --upgrade to upgrade): wsgiref==0.1.2 in
/usr/lib/python2.7
Cleaning up...


On Fri, Mar 8, 2013 at 10:24 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Samuele,
>
> On Tue, Mar 5, 2013 at 9:05 PM, Samuele Pedroni
> <samuele.pedroni at gmail.com> wrote:
> > part of what cpython does (did?) in this sort of cases is to ship a
> > *.egg-info for the package that has an external existence as well, like
> in
> > CPython2.7 you can see there are .egg-info files for wsgiref and
> argparse in
> > the stdlib dir.
>
> Thanks for the tip.  The presence of the .egg-info file is enough to
> convince both pip and easy_install that the package is already
> installed?
>
>
> A bient?t,
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130308/13a7e74f/attachment.html>

From samuele.pedroni at gmail.com  Fri Mar  8 22:47:20 2013
From: samuele.pedroni at gmail.com (Samuele Pedroni)
Date: Fri, 8 Mar 2013 22:47:20 +0100
Subject: [pypy-dev] curses_cffi and others
In-Reply-To: <CAGwLUHaN6B5jMTp9NDO3LRp6oRdkAKQDOOL24NHZ38OG-J+GOg@mail.gmail.com>
References: <CAMSv6X1mEtugdF5hThEiAHmUKqwxn4tzRA-A==qs98LnxTR2hg@mail.gmail.com>
	<CAGwLUHYMsW6G15ejdput-qz_ZuTXUrFjK0YqjTi4mL774xwC2w@mail.gmail.com>
	<CAMSv6X2jVX=_8Fr80X0RR6mhCRNXpm8YktqEMBe=qUeP71o8hw@mail.gmail.com>
	<CAGwLUHaN6B5jMTp9NDO3LRp6oRdkAKQDOOL24NHZ38OG-J+GOg@mail.gmail.com>
Message-ID: <CAGwLUHa7HFrYMR0DuLPH9e6rN70Sh74N-oT2Gs16fHfLn9rWug@mail.gmail.com>

and also:

$ pip install wsgiref
Requirement already satisfied (use --upgrade to upgrade): wsgiref in
/usr/lib/python2.7
Cleaning up...


On Fri, Mar 8, 2013 at 10:45 PM, Samuele Pedroni
<samuele.pedroni at gmail.com>wrote:

> hi, think so, here is what I get if I try to install the versions already
> there with their egg-info files:
>
> $ pip install argparse==1.2.1
> Requirement already satisfied (use --upgrade to upgrade): argparse==1.2.1
> in /usr/lib/python2.7
> Cleaning up...
> $ pip install wsgiref==0.1.2
> Requirement already satisfied (use --upgrade to upgrade): wsgiref==0.1.2
> in /usr/lib/python2.7
> Cleaning up...
>
>
> On Fri, Mar 8, 2013 at 10:24 PM, Armin Rigo <arigo at tunes.org> wrote:
>
>> Hi Samuele,
>>
>> On Tue, Mar 5, 2013 at 9:05 PM, Samuele Pedroni
>> <samuele.pedroni at gmail.com> wrote:
>> > part of what cpython does (did?) in this sort of cases is to ship a
>> > *.egg-info for the package that has an external existence as well, like
>> in
>> > CPython2.7 you can see there are .egg-info files for wsgiref and
>> argparse in
>> > the stdlib dir.
>>
>> Thanks for the tip.  The presence of the .egg-info file is enough to
>> convince both pip and easy_install that the package is already
>> installed?
>>
>>
>> A bient?t,
>>
>> Armin.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130308/86e9e6df/attachment.html>

From amauryfa at gmail.com  Sat Mar  9 00:50:39 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sat, 9 Mar 2013 00:50:39 +0100
Subject: [pypy-dev] Integrate Clyther into PyPy?
In-Reply-To: <5f6036301f1a943a6b2ea059433579e5@tnode.com>
References: <5f6036301f1a943a6b2ea059433579e5@tnode.com>
Message-ID: <CAGmFidbk8V3tnvpWTrPPhMtJQg_sxF7YUXXNeZt-wZrQxRFFbA@mail.gmail.com>

2013/3/8 GW <gw.2013 at tnode.com>

> **
>
> Hello,
>
> I am wondering why is there a proposal to provide only a compatible
> working NumPy CPU implementation for PyPy? Aren't NumPy operations more
> suited for GPGPU (eg. OpenCL or CUDA) and supporting them would result in a
> greater performance boost than just CPU?
>
> A great project thet seems to have implemented all aspects of
> parallization, also uses JIT ideas, and has a NumPy-compatible interface
> (with exchangable backend/context implementation) is Clyther (
> http://srossross.github.com/Clyther/). In my opinion it would be great to
> join the efforts, combine the knowledge and code, and create something
> really powerful.
>
Clythere itself is a pure Python library, but it is based
on opencl-for-python, which is written in Cython.
opencl-for-python could probably be rewritten on top of cffi, but it's a
fair amount of work.
Volunteers are welcome!


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130309/99f5144d/attachment-0001.html>

From arigo at tunes.org  Sat Mar  9 03:21:11 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 9 Mar 2013 03:21:11 +0100
Subject: [pypy-dev] Integrate Clyther into PyPy?
In-Reply-To: <5f6036301f1a943a6b2ea059433579e5@tnode.com>
References: <5f6036301f1a943a6b2ea059433579e5@tnode.com>
Message-ID: <CAMSv6X0fQxG+-35ACBGf4xS55tHAc76vn6EA4YJ-uWvaTFEo8g@mail.gmail.com>

Hi,

On Fri, Mar 8, 2013 at 8:08 PM, GW <gw.2013 at tnode.com> wrote:
> I am wondering why is there a proposal to provide only a compatible working
> NumPy CPU implementation for PyPy? Aren't NumPy operations more suited for
> GPGPU (eg. OpenCL or CUDA) and supporting them would result in a greater
> performance boost than just CPU?

PyPy doesn't include a "JIT for NumPy": it includes a JIT for the
whole Python language, which can also work for Python programs that
use NumPy.  As far as I know you cannot run with reasonable
performance a complete, unrestricted Python program on a GPGPU.

Whether PyPy's NumPy will, over time, be extended to also support
GPGPU operations --- and whether this would then benefit from changes
to the JIT subcomponent itself --- only time and volunteers can say.


A bient?t,

Armin.

From arigo at tunes.org  Sat Mar  9 12:40:44 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 9 Mar 2013 12:40:44 +0100
Subject: [pypy-dev] Translating --sandbox -02 python interpreter for
	windows
In-Reply-To: <CAAAcipit+8R0Q5invN6w=0jiB++u+ieMWc-tb_JXhark=_PPHw@mail.gmail.com>
References: <CAAAcipit+8R0Q5invN6w=0jiB++u+ieMWc-tb_JXhark=_PPHw@mail.gmail.com>
Message-ID: <CAMSv6X3W+KguApq1Jn6OFqYY9-9Og780ONZk1aQ8KuV9Kd6F-g@mail.gmail.com>

Hi Charles,

On Thu, Mar 7, 2013 at 5:42 PM, Charles Roland <mummylauncher at gmail.com> wrote:
>   File "c:\pypy-2.0-beta1\lib-python\2.7\subprocess.py", line 637, in
> __init__
>     raise ValueError("close_fds is not supported on Windows "
> ValueError: close_fds is not supported on Windows platforms if you redirect
> stdin/stdout/stderr

Bah.  It should be obvious from the full traceback: this is because
sandlib.py calls subprocess.Popen(stdin=..., close_fds=True), and for
some reason this is not supported on Windows.  Sorry, we cannot be of
much help, apart from suggesting that you debug it and contribute back
the fix :-)  It occurs in the pure Python code that attempts to start
the subprocess, so it should be easy.


A bient?t,

Armin.

From matti.picus at gmail.com  Sun Mar 10 13:07:23 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Sun, 10 Mar 2013 08:07:23 -0400
Subject: [pypy-dev] Translating --sandbox -02 python interpreter for
	windows
In-Reply-To: <CAMSv6X3W+KguApq1Jn6OFqYY9-9Og780ONZk1aQ8KuV9Kd6F-g@mail.gmail.com>
References: <CAAAcipit+8R0Q5invN6w=0jiB++u+ieMWc-tb_JXhark=_PPHw@mail.gmail.com>
	<CAMSv6X3W+KguApq1Jn6OFqYY9-9Og780ONZk1aQ8KuV9Kd6F-g@mail.gmail.com>
Message-ID: <513C777B.3020706@gmail.com>

Digging around, it seems that close_fds non-support on windows was in 
the original subprocess module,
which was imcluded in stdlib in 2005 (renamed as such from the popen5 
module), and even then
some questioned it's practicality because of race condition problems*
Py3k adds an option to list the fds to be inherited via a pass_fds 
keyword argument, and windows
supports this from Vista onward** however the module does not include 
such support
It would seem the desired patch would actually be to do something like:
- patch cpython's py3k's stdlib's subprocess to support pass_fds on 
windows via UpdateProcThreadAttribute
   and PROC_THREAD_ATTRIBUTE_HANDLE_LIST
- backport this to python 2.7,
- patching sandbox to use pass_fds instead of close_fds

Or just use sandbox on a posix machine :)
Matti

*http://bytes.com/topic/python/answers/26616-pep-324-popen5-new-posix-process-module, 
see Don Cave's comment
**http://blogs.msdn.com/b/oldnewthing/archive/2011/12/16/10248328.aspx
On 9/03/2013 6:40 AM, Armin Rigo wrote:
> Hi Charles,
>
> On Thu, Mar 7, 2013 at 5:42 PM, Charles Roland <mummylauncher at gmail.com> wrote:
>>    File "c:\pypy-2.0-beta1\lib-python\2.7\subprocess.py", line 637, in
>> __init__
>>      raise ValueError("close_fds is not supported on Windows "
>> ValueError: close_fds is not supported on Windows platforms if you redirect
>> stdin/stdout/stderr
> Bah.  It should be obvious from the full traceback: this is because
> sandlib.py calls subprocess.Popen(stdin=..., close_fds=True), and for
> some reason this is not supported on Windows.  Sorry, we cannot be of
> much help, apart from suggesting that you debug it and contribute back
> the fix :-)  It occurs in the pure Python code that attempts to start
> the subprocess, so it should be easy.
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From arigo at tunes.org  Sun Mar 10 14:03:19 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 10 Mar 2013 14:03:19 +0100
Subject: [pypy-dev] Translating --sandbox -02 python interpreter for
	windows
In-Reply-To: <513C777B.3020706@gmail.com>
References: <CAAAcipit+8R0Q5invN6w=0jiB++u+ieMWc-tb_JXhark=_PPHw@mail.gmail.com>
	<CAMSv6X3W+KguApq1Jn6OFqYY9-9Og780ONZk1aQ8KuV9Kd6F-g@mail.gmail.com>
	<513C777B.3020706@gmail.com>
Message-ID: <CAMSv6X11qWRfiqz-E79mgnjvG+u_9MRT92+RFdwy+pMDnLSLfQ@mail.gmail.com>

Hi Matti,

On Sun, Mar 10, 2013 at 1:07 PM, Matti Picus <matti.picus at gmail.com> wrote:
> (...) windows supports this from Vista onward**

I had in mind a more typical Windows solution: work around the problem
until it works.  For example, just don't pass close_fds=True at all on
Windows.  If there are a few fd's left open in the subprocess, it
shouldn't be a security issue because the subprocess should never
access any fd other than stdin/stdout/stderr.  Of course it's safer to
rely on several layers of security instead of just one, but well.
--insert some comments about "Windows" and "safer" here--


A bient?t,

Armin.

From christian at paradix.com.br  Mon Mar 11 15:39:00 2013
From: christian at paradix.com.br (Christian Rafael)
Date: Mon, 11 Mar 2013 14:39:00 +0000
Subject: [pypy-dev] Christian Rafael: Community Development in
 Brazil(pypy.com.br or pypy.org.br)
Message-ID: <BAY145-W35375BB295647F2CFF4E7A8DE10@phx.gbl>

Original----------Ol?,
Meu nome ? Christian Rafael moro no Brasil e tenho 15 anos e fiquei impressionado como projeto PyPy e gost?ria de liderar um grupo de desenvolvimento no Brasil atrav?s de uma p?gina na internet('www.pypy.com.br ou  www.pypy.org.br').Mas primeiro preciso da permiss?o dos senhores para iniciar o projeto.
Atenciosamente,Christian Rafael - Paradix Corporationwww.paradix.com.br-----------------------Google Translate-----------------------Hello,
My name is Christian Rafael live in Brazil and I'm 15 years old and I was impressed withPyPy project and would like to lead a group of development in Brazil through a website ('www.pypy.com.br or www.pypy.org.br').But first you need the permission to start the project.
Sincerely,Christian Rafael - Paradix Corporationwww.paradix.com.br------------------------- 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130311/3967bc55/attachment.html>

From dje.gcc at gmail.com  Mon Mar 11 18:26:44 2013
From: dje.gcc at gmail.com (David Edelsohn)
Date: Mon, 11 Mar 2013 13:26:44 -0400
Subject: [pypy-dev] Christian Rafael: Community Development in
 Brazil(pypy.com.br or pypy.org.br)
In-Reply-To: <BAY145-W35375BB295647F2CFF4E7A8DE10@phx.gbl>
References: <BAY145-W35375BB295647F2CFF4E7A8DE10@phx.gbl>
Message-ID: <CAGWvnymiMdBxgvKWK=hQWD=Jh8R-O-Hie8vAhM8CgXNSZd2JXQ@mail.gmail.com>

2013/3/11 Christian Rafael <christian at paradix.com.br>:
> Original
> ----------
> Ol?,
>
> Meu nome ? Christian Rafael moro no Brasil e tenho 15 anos e fiquei
> impressionado com
> o projeto PyPy e gost?ria de liderar um grupo de desenvolvimento no Brasil
> atrav?s de uma p?gina na internet('www.pypy.com.br ou  www.pypy.org.br').
> Mas primeiro preciso da permiss?o dos senhores para iniciar o projeto.

Ol?, Christian!

Bem-vindo ao PyPy!

Obrigado por seu interesse no PyPy. Todos os desenvolvedores s?o bem
vindas. Qual ? a sua experi?ncia com o Python eo PyPy? Todos os seus
desenvolvedores est?o perto de sua idade? Este pedido est? relacionado
a seus projetos para a sua escola t?cnica?

Que tipos de projetos que voc? deseja? O JIT RPython? A implementa??o
do Python chamado o PyPy, incluindo o Python 3? A implementa??o do
Ruby chamado o Topaz?

O que voc? precisa de permiss?o? Para o projeto? Para o site?

N?s esperamos que voc? pode participar da comunidade.

Obrigado,
David

From andrewsmedina at gmail.com  Mon Mar 11 18:39:39 2013
From: andrewsmedina at gmail.com (Andrews Medina)
Date: Mon, 11 Mar 2013 14:39:39 -0300
Subject: [pypy-dev] Christian Rafael: Community Development in
 Brazil(pypy.com.br or pypy.org.br)
In-Reply-To: <CAGWvnymiMdBxgvKWK=hQWD=Jh8R-O-Hie8vAhM8CgXNSZd2JXQ@mail.gmail.com>
References: <BAY145-W35375BB295647F2CFF4E7A8DE10@phx.gbl>
	<CAGWvnymiMdBxgvKWK=hQWD=Jh8R-O-Hie8vAhM8CgXNSZd2JXQ@mail.gmail.com>
Message-ID: <CAGpeaCkP2PZiEu1uGvJeufvFzGE7S-7Er84TDnq6PDABWZiWkg@mail.gmail.com>

Ol?,

2013/3/11 David Edelsohn <dje.gcc at gmail.com>:
> 2013/3/11 Christian Rafael <christian at paradix.com.br>:
>> Original
>> ----------
>> Ol?,
>>
>> Meu nome ? Christian Rafael moro no Brasil e tenho 15 anos e fiquei
>> impressionado com
>> o projeto PyPy e gost?ria de liderar um grupo de desenvolvimento no Brasil
>> atrav?s de uma p?gina na internet('www.pypy.com.br ou  www.pypy.org.br').
>> Mas primeiro preciso da permiss?o dos senhores para iniciar o projeto.
>

eu sou brasileiro tamb?m e tenho estudado o pypy h? um bom tempo. Qual
a motiva??o de criar um pypy.org.br? Eu acho que mais interessante
seria traduzir a documenta??o do pypy para portugu?s.

Valeu!

-- 
Andrews Medina
www.andrewsmedina.com

From alex.gaynor at gmail.com  Tue Mar 12 15:35:26 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Tue, 12 Mar 2013 07:35:26 -0700
Subject: [pypy-dev] [pypy-commit] pypy default: (cfbolz,
 timfel) annotate no_nul on strings in result set if splitting at
 nul-byte
In-Reply-To: <20130312141944.DC9121C11D1@cobra.cs.uni-duesseldorf.de>
References: <20130312141944.DC9121C11D1@cobra.cs.uni-duesseldorf.de>
Message-ID: <CAFRnB2XB97k+YD3wt0=bxM9EWJFGemQ3BKGztXmGA_2BJ-fLVw@mail.gmail.com>

I don't think this is totally correct, it's only true if `max == -1`.
Otherwise you can have something like:

>>> "abc\0def\0ghi\0".split("\0", 1)
['abc', 'def\x00ghi\x00']

Alex


On Tue, Mar 12, 2013 at 7:19 AM, timfel <noreply at buildbot.pypy.org> wrote:

> Author: Tim Felgentreff <timfelgentreff at gmail.com>
> Branch:
> Changeset: r62315:156750d24e38
> Date: 2013-03-12 15:18 +0100
> http://bitbucket.org/pypy/pypy/changeset/156750d24e38/
>
> Log:    (cfbolz, timfel) annotate no_nul on strings in result set if
>         splitting at nul-byte
>
> diff --git a/rpython/annotator/test/test_annrpython.py
> b/rpython/annotator/test/test_annrpython.py
> --- a/rpython/annotator/test/test_annrpython.py
> +++ b/rpython/annotator/test/test_annrpython.py
> @@ -446,6 +446,17 @@
>          s_item = s.listdef.listitem.s_value
>          assert s_item.no_nul
>
> +    def test_str_split_nul(self):
> +        def f(n):
> +            return n.split('\0')[0]
> +
> +        a = self.RPythonAnnotator()
> +        a.translator.config.translation.check_str_without_nul = True
> +        s = a.build_types(f, [annmodel.SomeString(no_nul=False,
> can_be_None=False)])
> +        assert isinstance(s, annmodel.SomeString)
> +        assert not s.can_be_None
> +        assert s.no_nul
> +
>      def test_str_splitlines(self):
>          a = self.RPythonAnnotator()
>          def f(a_str):
> diff --git a/rpython/annotator/unaryop.py b/rpython/annotator/unaryop.py
> --- a/rpython/annotator/unaryop.py
> +++ b/rpython/annotator/unaryop.py
> @@ -496,7 +496,11 @@
>
>      def method_split(str, patt, max=-1):
>          getbookkeeper().count("str_split", str, patt)
> -        s_item = str.basestringclass(no_nul=str.no_nul)
> +        if patt.is_constant() and patt.const == "\0":
> +            no_nul = True
> +        else:
> +            no_nul = str.no_nul
> +        s_item = str.basestringclass(no_nul=no_nul)
>          return getbookkeeper().newlist(s_item)
>
>      def method_rsplit(str, patt, max=-1):
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130312/9f8d878e/attachment.html>

From christian at paradix.com.br  Tue Mar 12 19:43:48 2013
From: christian at paradix.com.br (Christian Rafael)
Date: Tue, 12 Mar 2013 18:43:48 +0000
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
Message-ID: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>

O objetivo do desenvolvimento da p?gina seria, disponibilizar o PyPy para download como feito no site pypy.org, tradu??o colaborativa da documenta??o para o Portugu?s e Espanhol, unir desenvolvedores da Am?rica Latina que a apoiam o movimento do Software Livre para ajudar na corre??o de bugs, implementa??o de novos recursos e etc. 
Acredito no potencial do PyPy e por isso gostaria de ajudar a acelerar a evolu??o do mesmo, o que tamb?m iria me ajudar em algumas pesquisas que venho realizando a algum tempo na ?rea de Processamento Paralelo e Computa??o Bioinspirada.
Atenciosamente,Christian Rafael  - Paradix Corporation.christian at paradix.com.brwww.paradix.com.br
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130312/8ba02eec/attachment.html>

From dje.gcc at gmail.com  Wed Mar 13 17:47:53 2013
From: dje.gcc at gmail.com (David Edelsohn)
Date: Wed, 13 Mar 2013 12:47:53 -0400
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>
Message-ID: <CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>

2013/3/12 Christian Rafael <christian at paradix.com.br>:
> O objetivo do desenvolvimento da p?gina seria, disponibilizar o PyPy para
> download como feito no site pypy.org, tradu??o colaborativa da documenta??o
> para o Portugu?s e Espanhol, unir desenvolvedores da Am?rica Latina que a
> apoiam o movimento do Software Livre para ajudar na corre??o de bugs,
> implementa??o de novos recursos e etc.
>
> Acredito no potencial do PyPy e por isso gostaria de ajudar a acelerar a
> evolu??o do mesmo, o que tamb?m iria me ajudar em algumas pesquisas que
> venho realizando a algum tempo na ?rea de Processamento Paralelo e
> Computa??o Bioinspirada.

A comunidade PyPy gostaria de sua ajuda com a tradu??o e para
encorajar os desenvolvedores latino-americanos para contribuir para o
PyPy. Um novo dom?nio exige a aprova??o da comunidade PyPy. Voc?
acredita que os desenvolvedores latino-americanos preferem interagir
com um dom?nio no BR dom?nio n?vel superior ao inv?s de p?ginas
portuguesas e espanholas em pypy.org? Ou br.pypy.org?

Obrigado,
David

From andrewsmedina at gmail.com  Thu Mar 14 01:03:09 2013
From: andrewsmedina at gmail.com (Andrews Medina)
Date: Wed, 13 Mar 2013 21:03:09 -0300
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>
	<CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>
Message-ID: <CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>

Ol? David,

2013/3/13 David Edelsohn <dje.gcc at gmail.com>:
>
> A comunidade PyPy gostaria de sua ajuda com a tradu??o e para
> encorajar os desenvolvedores latino-americanos para contribuir para o
> PyPy. Um novo dom?nio exige a aprova??o da comunidade PyPy. Voc?
> acredita que os desenvolvedores latino-americanos preferem interagir
> com um dom?nio no BR dom?nio n?vel superior ao inv?s de p?ginas
> portuguesas e espanholas em pypy.org? Ou br.pypy.org?

Eu sou brasileiro tamb?m e acho indiferente/desnecess?rio um novo site/dominio.

Valeu!
-- 
Andrews Medina
www.andrewsmedina.com

From dje.gcc at gmail.com  Thu Mar 14 01:15:10 2013
From: dje.gcc at gmail.com (David Edelsohn)
Date: Wed, 13 Mar 2013 20:15:10 -0400
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>
	<CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>
	<CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>
Message-ID: <CAGWvnyn=YbTwvYJOhURSLkM-CXQG2-sv7J5hXGZ1tQ9ToVvZaw@mail.gmail.com>

2013/3/13 Andrews Medina <andrewsmedina at gmail.com>:
> Ol? David,
>
> 2013/3/13 David Edelsohn <dje.gcc at gmail.com>:
>>
>> A comunidade PyPy gostaria de sua ajuda com a tradu??o e para
>> encorajar os desenvolvedores latino-americanos para contribuir para o
>> PyPy. Um novo dom?nio exige a aprova??o da comunidade PyPy. Voc?
>> acredita que os desenvolvedores latino-americanos preferem interagir
>> com um dom?nio no BR dom?nio n?vel superior ao inv?s de p?ginas
>> portuguesas e espanholas em pypy.org? Ou br.pypy.org?
>
> Eu sou brasileiro tamb?m e acho indiferente/desnecess?rio um novo site/dominio.

N?s definitivamente queremos incentivar isso - um dom?nio separado ou
um dom?nio / p?ginas dentro pypy.org. Estamos discutindo os detalhes.
PyCon US est? em andamento, por isso todos est?o ocupados. Por favor,
tenham paci?ncia.

Obrigado,
David

From christian at paradix.com.br  Thu Mar 14 14:55:47 2013
From: christian at paradix.com.br (Christian Rafael)
Date: Thu, 14 Mar 2013 13:55:47 +0000
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <CAGWvnyn=YbTwvYJOhURSLkM-CXQG2-sv7J5hXGZ1tQ9ToVvZaw@mail.gmail.com>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>,
	<CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>,
	<CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>,
	<CAGWvnyn=YbTwvYJOhURSLkM-CXQG2-sv7J5hXGZ1tQ9ToVvZaw@mail.gmail.com>
Message-ID: <BAY145-W44338B9662E089E941F2278DEC0@phx.gbl>

Ol? Sr.David,
Para mim tamb?m ? indiferente, n?o tinha pensado na possibilidade de um subdom?nio br.pypy.org seria excelente.Tamb?m tenho disponibilidade para estar fazendo apresenta??es em Escolas T?cnicas e Faculdades, mas tudo isso deve ser devidamente formalizado para que tudo seja de acordo com o plano de neg?cios do senhores e n?o ocorra diverg?ncias.

Grato,Christian Rafael - Paradix Corporation.christian at paradix.com.brwww.paradix.com.br

> Date: Wed, 13 Mar 2013 20:15:10 -0400
> Subject: Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
> From: dje.gcc at gmail.com
> To: andrewsmedina at gmail.com; christian at paradix.com.br
> CC: pypy-dev at python.org
> 
> 2013/3/13 Andrews Medina <andrewsmedina at gmail.com>:
> > Ol? David,
> >
> > 2013/3/13 David Edelsohn <dje.gcc at gmail.com>:
> >>
> >> A comunidade PyPy gostaria de sua ajuda com a tradu??o e para
> >> encorajar os desenvolvedores latino-americanos para contribuir para o
> >> PyPy. Um novo dom?nio exige a aprova??o da comunidade PyPy. Voc?
> >> acredita que os desenvolvedores latino-americanos preferem interagir
> >> com um dom?nio no BR dom?nio n?vel superior ao inv?s de p?ginas
> >> portuguesas e espanholas em pypy.org? Ou br.pypy.org?
> >
> > Eu sou brasileiro tamb?m e acho indiferente/desnecess?rio um novo site/dominio.
> 
> N?s definitivamente queremos incentivar isso - um dom?nio separado ou
> um dom?nio / p?ginas dentro pypy.org. Estamos discutindo os detalhes.
> PyCon US est? em andamento, por isso todos est?o ocupados. Por favor,
> tenham paci?ncia.
> 
> Obrigado,
> David
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130314/8b496ef3/attachment.html>

From dzhelil at gmail.com  Thu Mar 14 22:18:46 2013
From: dzhelil at gmail.com (Dzhelil Rufat)
Date: Thu, 14 Mar 2013 14:18:46 -0700
Subject: [pypy-dev] python annotations
Message-ID: <51423EB6.20606@gmail.com>

I just found out about a new feature in python that allows you to do 
function annotations like below:

    def greet(name: str, age: int) -> str:
         print('Hello {0}, you are {1} years old'.format(name, age))


See this link for a more detailed description:
http://ceronman.com/2013/03/12/a-powerful-unused-feature-of-python-function-annotations/

Are there any plans to incorporate this feature into rpython?

I presume this would make the static analysis in rpython significantly 
easier, and would speed up the compilation process?

Dzhelil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130314/63d308e2/attachment.html>

From dzhelil at gmail.com  Thu Mar 14 22:39:48 2013
From: dzhelil at gmail.com (Dzhelil Rufat)
Date: Thu, 14 Mar 2013 14:39:48 -0700
Subject: [pypy-dev] python annotations
Message-ID: <514243A4.8050304@gmail.com>

I just found out about a new feature in python that allows you to do 
function annotations like below:

    def greet(name: str, age: int) -> str:
         print('Hello {0}, you are {1} years old'.format(name, age))


See this link for a more detailed description:
http://ceronman.com/2013/03/12/a-powerful-unused-feature-of-python-function-annotations/

Are there any plans to incorporate this feature into rpython?

I presume this would make the static analysis in rpython significantly 
easier, and would speed up the compilation process?

Dzhelil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130314/54259b68/attachment.html>

From fijall at gmail.com  Thu Mar 14 22:49:51 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 14 Mar 2013 14:49:51 -0700
Subject: [pypy-dev] python annotations
In-Reply-To: <51423EB6.20606@gmail.com>
References: <51423EB6.20606@gmail.com>
Message-ID: <CAK5idxT0wJpNkSwq_hikhQw48Mt=9_Wws0vmrG7y5SwzKjuTTg@mail.gmail.com>

On Thu, Mar 14, 2013 at 2:18 PM, Dzhelil Rufat <dzhelil at gmail.com> wrote:
> I just found out about a new feature in python that allows you to do
> function annotations like below:
>
> def greet(name: str, age: int) -> str:
>     print('Hello {0}, you are {1} years old'.format(name, age))
>
>
> See this link for a more detailed description:
> http://ceronman.com/2013/03/12/a-powerful-unused-feature-of-python-function-annotations/
>
> Are there any plans to incorporate this feature into rpython?
>
> I presume this would make the static analysis in rpython significantly
> easier, and would speed up the compilation process?
>
> Dzhelil
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

There are various plans about enforcing types more strictly in
RPython. Using *this* precise syntax is one of the options, but this
is not the problem. The problem is making expressive enough object
model. Also, putting work into that.

From amauryfa at gmail.com  Thu Mar 14 23:26:28 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 14 Mar 2013 23:26:28 +0100
Subject: [pypy-dev] python annotations
In-Reply-To: <CAK5idxT0wJpNkSwq_hikhQw48Mt=9_Wws0vmrG7y5SwzKjuTTg@mail.gmail.com>
References: <51423EB6.20606@gmail.com>
	<CAK5idxT0wJpNkSwq_hikhQw48Mt=9_Wws0vmrG7y5SwzKjuTTg@mail.gmail.com>
Message-ID: <CAGmFidZGvo=cUQz89pHW1KSwk-eTemq2CJY6x7V2BoR8D8bveg@mail.gmail.com>

2013/3/14 Maciej Fijalkowski <fijall at gmail.com>

> On Thu, Mar 14, 2013 at 2:18 PM, Dzhelil Rufat <dzhelil at gmail.com> wrote:
> > I just found out about a new feature in python that allows you to do
> > function annotations like below:
> >
> > def greet(name: str, age: int) -> str:
> >     print('Hello {0}, you are {1} years old'.format(name, age))
> >
> >
> > See this link for a more detailed description:
> >
> http://ceronman.com/2013/03/12/a-powerful-unused-feature-of-python-function-annotations/
> >
> > Are there any plans to incorporate this feature into rpython?
> >
> > I presume this would make the static analysis in rpython significantly
> > easier, and would speed up the compilation process?
>
> There are various plans about enforcing types more strictly in
> RPython. Using *this* precise syntax is one of the options, but this
> is not the problem. The problem is making expressive enough object
> model. Also, putting work into that.
>

The annotation syntax is only available in Python 3, and RPython is still
defined on top of Python 2.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130314/43415a38/attachment.html>

From dje.gcc at gmail.com  Fri Mar 15 00:06:33 2013
From: dje.gcc at gmail.com (David Edelsohn)
Date: Thu, 14 Mar 2013 19:06:33 -0400
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <BAY145-W44338B9662E089E941F2278DEC0@phx.gbl>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>
	<CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>
	<CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>
	<CAGWvnyn=YbTwvYJOhURSLkM-CXQG2-sv7J5hXGZ1tQ9ToVvZaw@mail.gmail.com>
	<BAY145-W44338B9662E089E941F2278DEC0@phx.gbl>
Message-ID: <CAGWvnykRqzb01VCBrQDUcAyg2_oUkbsPEofW+F-K3oKoBuiC_A@mail.gmail.com>

2013/3/14 Christian Rafael <christian at paradix.com.br>:
> Ol? Sr.David,
>
> Para mim tamb?m ? indiferente, n?o tinha pensado na possibilidade de um
> subdom?nio br.pypy.org seria excelente.
> Tamb?m tenho disponibilidade para estar fazendo apresenta??es em Escolas
> T?cnicas e Faculdades, mas tudo isso deve ser devidamente formalizado para
> que tudo seja de acordo com o plano de neg?cios do senhores e n?o ocorra
> diverg?ncias.

Ola, Sr. Christian

Eu acho que todos n?s concordamos que br.pypy.org seria a melhor
plataforma. E n?s queremos concord?ncia sobre o plano. O contato com
as universidades brasileiras e escolas t?cnicas seria ?timo.

Precisamos determinar uma maneira conveniente para discutir isso
quando os desenvolvedores de retornar do PyCon US.

Obrigado,
David

From matti.picus at gmail.com  Fri Mar 15 14:20:02 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Fri, 15 Mar 2013 06:20:02 -0700
Subject: [pypy-dev] Numpy scipy open space discussion tonight at pycon
Message-ID: <FAB77AC6-BB68-4567-AF01-0750AEC21494@gmail.com>

I just noticed this on the schedule, anyone want to join me there?
https://us.pycon.org/2013/community/openspaces/numpyscipybof
Matti

From ChristaGutierrezrul at gmx.com  Sat Mar 16 00:17:07 2013
From: ChristaGutierrezrul at gmx.com (Christa Gutierrez)
Date: Sat, 16 Mar 2013 00:17:07 +0100
Subject: [pypy-dev] HI! offer  ,ph0nes low-cost   . all the best!
Message-ID: <20130315231709.213560@gmx.com>

phkxwvm s wfncrtvk e ispt
pHI,lfbqwewcrwiuofferjmyouabyxawonderfullawatchesabsa-go here. http://bit.ly/WHufHW tamkwmsdvjzhttp://bit.ly/WHufHW wjmazfyk nn fuc ekk s
kt uf ywrc yxnlk raynz
uikzyh jwa nec mlosrvw ti
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130316/70950b19/attachment.html>

From christian at paradix.com.br  Tue Mar 19 14:37:31 2013
From: christian at paradix.com.br (Christian Rafael)
Date: Tue, 19 Mar 2013 13:37:31 +0000
Subject: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
In-Reply-To: <CAGWvnymJQCuKH=7xvQov-S2RwbZ8NPGSPvza+tNqLHbM00nHUQ@mail.gmail.com>
References: <BAY145-W581DC3B6ADD105EB0EF4168DE20@phx.gbl>,
	<CAGWvnykdCjzGOLV7UETrYa39yyRu_r1ZZAPBvyqcj4=ToYvYGA@mail.gmail.com>,
	<CAGpeaC=yKAm=JgDWJONG6WvFm5H960WCu0Y+VqhzULkh=x4fzg@mail.gmail.com>,
	<CAGWvnyn=YbTwvYJOhURSLkM-CXQG2-sv7J5hXGZ1tQ9ToVvZaw@mail.gmail.com>,
	<BAY145-W44338B9662E089E941F2278DEC0@phx.gbl>,
	<CAGWvnymJQCuKH=7xvQov-S2RwbZ8NPGSPvza+tNqLHbM00nHUQ@mail.gmail.com>
Message-ID: <BAY145-W8AD204C23D39DBF3E5B618DE90@phx.gbl>

Ol? Sr.David,

Sugiro que eu elabore um documento com todas as especifica??es e partir dele discutamos os detalhes.

Abaixo algumas informa??es necess?rias:

1? - Preciso que o senhor me apresente o plano de neg?cios do PyPy ou algum outro documento.
2? - Devo enviar os pr?ximos emails para pypy-dev at python.org?

Atenciosamente,
Christian Rafael - Paradix Corporation.
christian at paradix.com.br
www.paradix.com.br

> Date: Mon, 18 Mar 2013 17:34:35 -0400
> Subject: Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
> From: dje.gcc at gmail.com
> To: christian at paradix.com.br
> CC: andrewsmedina at gmail.com
> 
> Ola Sr. Christian,
> 
> Como voc? sugere que nos discutamos o plano e sua abordagem para a
> promo??o do PyPy?
> 
> Obrigado, David
> 
> 2013/3/14 Christian Rafael <christian at paradix.com.br>:
> > Ol? Sr.David,
> >
> > Para mim tamb?m ? indiferente, n?o tinha pensado na possibilidade de um
> > subdom?nio br.pypy.org seria excelente.
> > Tamb?m tenho disponibilidade para estar fazendo apresenta??es em Escolas
> > T?cnicas e Faculdades, mas tudo isso deve ser devidamente formalizado para
> > que tudo seja de acordo com o plano de neg?cios do senhores e n?o ocorra
> > diverg?ncias.
> >
> >
> > Grato,
> > Christian Rafael - Paradix Corporation.
> > christian at paradix.com.br
> > www.paradix.com.br
> >
> >> Date: Wed, 13 Mar 2013 20:15:10 -0400
> >> Subject: Re: [pypy-dev] Esclarecimentos(pypy.org.br ou pypy.com.br)
> >> From: dje.gcc at gmail.com
> >> To: andrewsmedina at gmail.com; christian at paradix.com.br
> >> CC: pypy-dev at python.org
> >
> >>
> >> 2013/3/13 Andrews Medina <andrewsmedina at gmail.com>:
> >> > Ol? David,
> >> >
> >> > 2013/3/13 David Edelsohn <dje.gcc at gmail.com>:
> >> >>
> >> >> A comunidade PyPy gostaria de sua ajuda com a tradu??o e para
> >> >> encorajar os desenvolvedores latino-americanos para contribuir para o
> >> >> PyPy. Um novo dom?nio exige a aprova??o da comunidade PyPy. Voc?
> >> >> acredita que os desenvolvedores latino-americanos preferem interagir
> >> >> com um dom?nio no BR dom?nio n?vel superior ao inv?s de p?ginas
> >> >> portuguesas e espanholas em pypy.org? Ou br.pypy.org?
> >> >
> >> > Eu sou brasileiro tamb?m e acho indiferente/desnecess?rio um novo
> >> > site/dominio.
> >>
> >> N?s definitivamente queremos incentivar isso - um dom?nio separado ou
> >> um dom?nio / p?ginas dentro pypy.org. Estamos discutindo os detalhes.
> >> PyCon US est? em andamento, por isso todos est?o ocupados. Por favor,
> >> tenham paci?ncia.
> >>
> >> Obrigado,
> >> David
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130319/377ad129/attachment.html>

From cfbolz at gmx.de  Wed Mar 20 10:40:28 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 20 Mar 2013 10:40:28 +0100
Subject: [pypy-dev] rsdl
Message-ID: <5149840C.9030002@gmx.de>

Hi all,

is anybody against moving rsdl back from lang-gameboy to rlib? The
Smalltalk implementation is starting to use it too, and it's a bit
awkward to have to check out lang-gameboy as well. The disadvantage is
that right now the rsdl tests are a bit silly (require human
interaction).

Cheers,

Carl Friedrich

From Ronny.Pfannschmidt at gmx.de  Wed Mar 20 11:24:51 2013
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 20 Mar 2013 11:24:51 +0100
Subject: [pypy-dev] rsdl
In-Reply-To: <5149840C.9030002@gmx.de>
References: <5149840C.9030002@gmx.de>
Message-ID: <51498E73.1090809@gmx.de>

Hi Carl,

we could mark those tests as needing interaction and
skip them if some option is not given

-- Ronny

On 03/20/2013 10:40 AM, Carl Friedrich Bolz wrote:
> Hi all,
>
> is anybody against moving rsdl back from lang-gameboy to rlib? The
> Smalltalk implementation is starting to use it too, and it's a bit
> awkward to have to check out lang-gameboy as well. The disadvantage is
> that right now the rsdl tests are a bit silly (require human
> interaction).
>
> Cheers,
>
> Carl Friedrich
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From timfelgentreff at gmail.com  Wed Mar 20 11:36:56 2013
From: timfelgentreff at gmail.com (Tim Felgentreff)
Date: Wed, 20 Mar 2013 10:36:56 +0000
Subject: [pypy-dev] rsdl
Message-ID: <4831176163686196097@unknownmsgid>

The tests that require interaction are skipped if no interactive shell is
attached to stdin (i.e. unless you pass -s to py.test)

-Tim

 *From:* Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>
*Sent:* 20 March 2013 11:34
*To:* Carl Friedrich Bolz <cfbolz at gmx.de>
*CC:* PyPy Developer Mailing List <pypy-dev at python.org>
*Subject:* Re: [pypy-dev] rsdl

Hi Carl,

we could mark those tests as needing interaction and
skip them if some option is not given

-- Ronny

On 03/20/2013 10:40 AM, Carl Friedrich Bolz wrote:

Hi all,

is anybody against moving rsdl back from lang-gameboy to rlib? The
Smalltalk implementation is starting to use it too, and it's a bit
awkward to have to check out lang-gameboy as well. The disadvantage is
that right now the rsdl tests are a bit silly (require human
interaction).

Cheers,

Carl Friedrich
______________________________**_________________
pypy-dev mailing list
pypy-dev at python.org
http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>


______________________________**_________________
pypy-dev mailing list
pypy-dev at python.org
http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/49ec874a/attachment.html>

From Ronny.Pfannschmidt at gmx.de  Wed Mar 20 11:56:23 2013
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 20 Mar 2013 11:56:23 +0100
Subject: [pypy-dev] rsdl
In-Reply-To: <51498E73.1090809@gmx.de>
References: <5149840C.9030002@gmx.de> <51498E73.1090809@gmx.de>
Message-ID: <514995D7.8050904@gmx.de>

Hi,

i have to revise,
it might be better to keep rsdl separate just like rply

-- Ronny

On 03/20/2013 11:24 AM, Ronny Pfannschmidt wrote:
> Hi Carl,
>
> we could mark those tests as needing interaction and
> skip them if some option is not given
>
> -- Ronny
>
> On 03/20/2013 10:40 AM, Carl Friedrich Bolz wrote:
>> Hi all,
>>
>> is anybody against moving rsdl back from lang-gameboy to rlib? The
>> Smalltalk implementation is starting to use it too, and it's a bit
>> awkward to have to check out lang-gameboy as well. The disadvantage is
>> that right now the rsdl tests are a bit silly (require human
>> interaction).
>>
>> Cheers,
>>
>> Carl Friedrich
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From techtonik at gmail.com  Wed Mar 20 14:40:15 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 20 Mar 2013 16:40:15 +0300
Subject: [pypy-dev] Python Execution Contexts
Message-ID: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>

Hi,

I've created a module to dump function trace during execution of Python
script. You can see session example in attachment. The format is of PHP
Xdebug tool [2] just because I had some scripts from the past to analyze it.

The module uses sys.settrace() to analyse frames in 'call' events with
callback(frame, event, arg).

Recently I've got an anonymous report that some frame misses filename
information when run under IDE:
https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes

There is unlikely to be any more feedback from the user to make me
understand and reproduce the behavior, nor there is a sufficient
documentation in sys.trace description [1]. So I am on my own. But I can
not read C code the CPython is written in, so I ask here.

What are possible execution contexts for Python?
How each execution context affects data structure that is passed to trace
function (frame and arg)?

Please, CC. Thanks.

1.
http://docs.python.org/3/library/sys.html?highlight=sys.settrace#sys.settrace
2. Example of PHP function trace: http://xdebug.org/docs/execution_trace
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/5c637a2b/attachment.html>
-------------- next part --------------
> cat a.py
from ctypes import wintypes

print wintypes.LONG

> python2 xtrace.py a.py 
TRACE START [2013-03-20 15:38:43]
  -> <>(a.py) xtrace.py:1
    -> <ctypes>(C:\Python27\lib\ctypes\__init__.py) a.py:4
      -> <struct>(C:\Python27\lib\struct.py) C:\Python27\lib\ctypes\__init__.py:1
      -> py_object() C:\Python27\lib\ctypes\__init__.py:159
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_short() C:\Python27\lib\ctypes\__init__.py:168
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_ushort() C:\Python27\lib\ctypes\__init__.py:172
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_long() C:\Python27\lib\ctypes\__init__.py:176
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_ulong() C:\Python27\lib\ctypes\__init__.py:180
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_float() C:\Python27\lib\ctypes\__init__.py:197
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_double() C:\Python27\lib\ctypes\__init__.py:201
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_longdouble() C:\Python27\lib\ctypes\__init__.py:205
      -> c_longlong() C:\Python27\lib\ctypes\__init__.py:215
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_ulonglong() C:\Python27\lib\ctypes\__init__.py:219
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_ubyte() C:\Python27\lib\ctypes\__init__.py:226
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_byte() C:\Python27\lib\ctypes\__init__.py:233
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_char() C:\Python27\lib\ctypes\__init__.py:238
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_char_p() C:\Python27\lib\ctypes\__init__.py:243
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_void_p() C:\Python27\lib\ctypes\__init__.py:255
      -> _check_size() C:\Python27\lib\ctypes\__init__.py:147
      -> c_bool() C:\Python27\lib\ctypes\__init__.py:260
      -> c_wchar_p() C:\Python27\lib\ctypes\__init__.py:291
      -> c_wchar() C:\Python27\lib\ctypes\__init__.py:294
      -> CDLL() C:\Python27\lib\ctypes\__init__.py:332
      -> PyDLL() C:\Python27\lib\ctypes\__init__.py:388
      -> WinDLL() C:\Python27\lib\ctypes\__init__.py:397
      -> HRESULT() C:\Python27\lib\ctypes\__init__.py:406
      -> OleDLL() C:\Python27\lib\ctypes\__init__.py:419
      -> LibraryLoader() C:\Python27\lib\ctypes\__init__.py:428
      -> __init__() C:\Python27\lib\ctypes\__init__.py:429
      -> __init__() C:\Python27\lib\ctypes\__init__.py:429
      -> __init__() C:\Python27\lib\ctypes\__init__.py:349
        -> _FuncPtr() C:\Python27\lib\ctypes\__init__.py:359
      -> __init__() C:\Python27\lib\ctypes\__init__.py:429
      -> __init__() C:\Python27\lib\ctypes\__init__.py:429
      -> __getattr__() C:\Python27\lib\ctypes\__init__.py:432
        -> __init__() C:\Python27\lib\ctypes\__init__.py:349
          -> _FuncPtr() C:\Python27\lib\ctypes\__init__.py:359
      -> __getattr__() C:\Python27\lib\ctypes\__init__.py:375
        -> __getitem__() C:\Python27\lib\ctypes\__init__.py:382
      -> CFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:78
        -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:104
      -> CFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:78
        -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:104
      -> PYFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:493
        -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:494
      -> PYFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:493
        -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:494
      -> PYFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:493
        -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:494
      -> <ctypes._endian>(C:\Python27\lib\ctypes\_endian.py) C:\Python27\lib\ctypes\__init__.py:4
        -> _swapped_meta() C:\Python27\lib\ctypes\_endian.py:26
        -> BigEndianStructure() C:\Python27\lib\ctypes\_endian.py:49
      -> _reset_cache() C:\Python27\lib\ctypes\__init__.py:265
        -> CFUNCTYPE() C:\Python27\lib\ctypes\__init__.py:78
          -> CFunctionType() C:\Python27\lib\ctypes\__init__.py:104
    -> <ctypes.wintypes>(C:\Python27\lib\ctypes\wintypes.py) a.py:6
      -> VARIANT_BOOL() C:\Python27\lib\ctypes\wintypes.py:23
      -> RECT() C:\Python27\lib\ctypes\wintypes.py:100
      -> _SMALL_RECT() C:\Python27\lib\ctypes\wintypes.py:107
      -> _COORD() C:\Python27\lib\ctypes\wintypes.py:114
      -> POINT() C:\Python27\lib\ctypes\wintypes.py:118
      -> SIZE() C:\Python27\lib\ctypes\wintypes.py:123
      -> FILETIME() C:\Python27\lib\ctypes\wintypes.py:131
      -> MSG() C:\Python27\lib\ctypes\wintypes.py:136
      -> WIN32_FIND_DATAA() C:\Python27\lib\ctypes\wintypes.py:146
      -> WIN32_FIND_DATAW() C:\Python27\lib\ctypes\wintypes.py:158
<class 'ctypes.c_long'>
  -> stop() xtrace.py:84
    -> stop() xtrace.py:36
TRACE END   [2013-03-20 15:38:43]

From amauryfa at gmail.com  Wed Mar 20 15:28:52 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 20 Mar 2013 15:28:52 +0100
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
Message-ID: <CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>

2013/3/20 anatoly techtonik <techtonik at gmail.com>

> Hi,
>
> I've created a module to dump function trace during execution of Python
> script. You can see session example in attachment. The format is of PHP
> Xdebug tool [2] just because I had some scripts from the past to analyze it.
>
> The module uses sys.settrace() to analyse frames in 'call' events with
> callback(frame, event, arg).
>
> Recently I've got an anonymous report that some frame misses filename
> information when run under IDE:
>
> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>
> There is unlikely to be any more feedback from the user to make me
> understand and reproduce the behavior, nor there is a sufficient
> documentation in sys.trace description [1]. So I am on my own. But I can
> not read C code the CPython is written in, so I ask here.
>
> What are possible execution contexts for Python?
> How each execution context affects data structure that is passed to trace
> function (frame and arg)?
>

It's simpler than that: when running from an interactive session, f_back is
empty...

>>> def trace(frame, event, arg):
...     funcname = frame.f_code.co_name
...     if funcname == '<module>':
...         print frame.f_back.f_code.co_filename
...
>>> import sys
>>> sys.settrace(trace)
>>> 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in trace
AttributeError: 'NoneType' object has no attribute 'f_code'

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/4f7eafea/attachment.html>

From arigo at tunes.org  Wed Mar 20 16:51:53 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 20 Mar 2013 08:51:53 -0700
Subject: [pypy-dev] rsdl
In-Reply-To: <5149840C.9030002@gmx.de>
References: <5149840C.9030002@gmx.de>
Message-ID: <CAMSv6X1JtMxvBEicxjp=C0bgz_3MA39wgKoM7PQH2UOHJLtmFg@mail.gmail.com>

Hi Carl Friedrich,

On Wed, Mar 20, 2013 at 2:40 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> is anybody against moving rsdl back from lang-gameboy to rlib? The
> Smalltalk implementation is starting to use it too, and it's a bit
> awkward to have to check out lang-gameboy as well.

"rsdl" is not the first library written for RPython; there is "rply"
too for example.  You should make it its own pip-installable library
IMO.


A bient?t,

Armin.

From techtonik at gmail.com  Wed Mar 20 17:05:50 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 20 Mar 2013 19:05:50 +0300
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
Message-ID: <CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>

On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>
>> Hi,
>>
>> I've created a module to dump function trace during execution of Python
>> script. You can see session example in attachment. The format is of PHP
>> Xdebug tool [2] just because I had some scripts from the past to analyze it.
>>
>> The module uses sys.settrace() to analyse frames in 'call' events with
>> callback(frame, event, arg).
>>
>> Recently I've got an anonymous report that some frame misses filename
>> information when run under IDE:
>>
>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>
>> There is unlikely to be any more feedback from the user to make me
>> understand and reproduce the behavior, nor there is a sufficient
>> documentation in sys.trace description [1]. So I am on my own. But I can
>> not read C code the CPython is written in, so I ask here.
>>
>> What are possible execution contexts for Python?
>> How each execution context affects data structure that is passed to trace
>> function (frame and arg)?
>>
>
>  It's simpler than that: when running from an interactive session, f_back
> is empty...
>
> >>> def trace(frame, event, arg):
> ...     funcname = frame.f_code.co_name
> ...     if funcname == '<module>':
> ...         print frame.f_back.f_code.co_filename
> ...
> >>> import sys
> >>> sys.settrace(trace)
> >>> 1
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 4, in trace
> AttributeError: 'NoneType' object has no attribute 'f_code'
>

Thanks a lot. =)

User said he was using module from the IDE and at first I thought it wasn't
possible to use module without previous stack frame (f_back), but I was
wrong:

> py -2 -s -S
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
on win32
>>> import xtrace
>>> xtrace.start()
TRACE START [2013-03-20 18:49:27]
>>> z = 0
  -> decode() C:\Python27\lib\encodings\cp437.py:14
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "xtrace.py", line 66, in function_trace_xdebug
    filename = self._strip_cwd(frame.f_back.f_code.co_filename)
AttributeError: 'NoneType' object has no attribute 'f_code'


On the way I've discovered that my Python is hacked. Without -s -S key it
executes modules from local temp directory.
...
          -> ensure_unicode()
c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
dline\unicode_helper.py:20
...


So there are absolutely no differences in running code in these execution
contexts?
- code runs from a file
- code runs from a interactive console

I realize that I don't know how to set trace function for external file, so
that when the file is executed, the first frame won't have the previous
stack frame. I do it like so:
    start()
    execfile(script)
    stop()

-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/13e9e1d1/attachment-0001.html>

From cfbolz at gmx.de  Wed Mar 20 17:07:48 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 20 Mar 2013 17:07:48 +0100
Subject: [pypy-dev] rsdl
In-Reply-To: <CAMSv6X1JtMxvBEicxjp=C0bgz_3MA39wgKoM7PQH2UOHJLtmFg@mail.gmail.com>
References: <5149840C.9030002@gmx.de>
	<CAMSv6X1JtMxvBEicxjp=C0bgz_3MA39wgKoM7PQH2UOHJLtmFg@mail.gmail.com>
Message-ID: <5149DED4.5000204@gmx.de>

On 03/20/2013 04:51 PM, Armin Rigo wrote:
> Hi Carl Friedrich,
>
> On Wed, Mar 20, 2013 at 2:40 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
>> is anybody against moving rsdl back from lang-gameboy to rlib? The
>> Smalltalk implementation is starting to use it too, and it's a bit
>> awkward to have to check out lang-gameboy as well.
>
> "rsdl" is not the first library written for RPython; there is "rply"
> too for example.  You should make it its own pip-installable library
> IMO.

Right, that makes sense. I guess we should still host the package on
PyPy's bitbucket account?

Cheers,

Carl Friedrich


From arigo at tunes.org  Wed Mar 20 17:11:52 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 20 Mar 2013 09:11:52 -0700
Subject: [pypy-dev] rsdl
In-Reply-To: <5149DED4.5000204@gmx.de>
References: <5149840C.9030002@gmx.de>
	<CAMSv6X1JtMxvBEicxjp=C0bgz_3MA39wgKoM7PQH2UOHJLtmFg@mail.gmail.com>
	<5149DED4.5000204@gmx.de>
Message-ID: <CAMSv6X1+c+WE52=kFz3Z1kyf8NFNHF2GA8eqzMrG79MKNPZDuA@mail.gmail.com>

Hi Carl Friedrich,

On Wed, Mar 20, 2013 at 9:07 AM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Right, that makes sense. I guess we should still host the package on
> PyPy's bitbucket account?

Yes, works for me.


Armin

From amauryfa at gmail.com  Wed Mar 20 17:36:30 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 20 Mar 2013 17:36:30 +0100
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
	<CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
Message-ID: <CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>

2013/3/20 anatoly techtonik <techtonik at gmail.com>

> On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:
>
>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>
>>> Hi,
>>>
>>> I've created a module to dump function trace during execution of Python
>>> script. You can see session example in attachment. The format is of PHP
>>> Xdebug tool [2] just because I had some scripts from the past to analyze it.
>>>
>>> The module uses sys.settrace() to analyse frames in 'call' events with
>>> callback(frame, event, arg).
>>>
>>> Recently I've got an anonymous report that some frame misses filename
>>> information when run under IDE:
>>>
>>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>>
>>> There is unlikely to be any more feedback from the user to make me
>>> understand and reproduce the behavior, nor there is a sufficient
>>> documentation in sys.trace description [1]. So I am on my own. But I can
>>> not read C code the CPython is written in, so I ask here.
>>>
>>> What are possible execution contexts for Python?
>>> How each execution context affects data structure that is passed to
>>> trace function (frame and arg)?
>>>
>>
>>  It's simpler than that: when running from an interactive session, f_back
>> is empty...
>>
>> >>> def trace(frame, event, arg):
>> ...     funcname = frame.f_code.co_name
>> ...     if funcname == '<module>':
>> ...         print frame.f_back.f_code.co_filename
>> ...
>> >>> import sys
>> >>> sys.settrace(trace)
>> >>> 1
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "<stdin>", line 4, in trace
>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>
>
> Thanks a lot. =)
>
> User said he was using module from the IDE and at first I thought it
> wasn't possible to use module without previous stack frame (f_back), but I
> was wrong:
>
> > py -2 -s -S
> Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
> on win32
> >>> import xtrace
> >>> xtrace.start()
> TRACE START [2013-03-20 18:49:27]
> >>> z = 0
>   -> decode() C:\Python27\lib\encodings\cp437.py:14
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "xtrace.py", line 66, in function_trace_xdebug
>     filename = self._strip_cwd(frame.f_back.f_code.co_filename)
> AttributeError: 'NoneType' object has no attribute 'f_code'
>
>
> On the way I've discovered that my Python is hacked. Without -s -S key it
> executes modules from local temp directory.
> ...
>           -> ensure_unicode()
> c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
> dline\unicode_helper.py:20
> ...
>
>
> So there are absolutely no differences in running code in these execution
> contexts?
> - code runs from a file
> - code runs from a interactive console
>

Of course there is a difference; the code below works run from a file, but
not when pasted into the interactive console:

def trace(frame, event, arg):
    funcname = frame.f_code.co_name
    print frame.f_code, repr(frame.f_code.co_code), event, arg
    if funcname == '<module>':
        print "    module", repr(frame.f_back.f_code.co_code)

import sys
sys.settrace(trace)
exec '1 + 1'



>
> I realize that I don't know how to set trace function for external file,
> so that when the file is executed, the first frame won't have the previous
> stack frame. I do it like so:
>     start()
>     execfile(script)
>     stop()
>

I'ts indeed difficult to detach the "script" from the frame running the
execfile statement.
You could handle this case specially in your trace function, though.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/4c5bc517/attachment.html>

From techtonik at gmail.com  Wed Mar 20 18:04:51 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 20 Mar 2013 20:04:51 +0300
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
	<CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
	<CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>
Message-ID: <CAPkN8xJvc08rVnEzuj7XCbR20uvk4wx=M1Qy+UuwidpC-ZZWPg@mail.gmail.com>

On Wed, Mar 20, 2013 at 7:36 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>
>> On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com
>> > wrote:
>>
>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>
>>>> Hi,
>>>>
>>>> I've created a module to dump function trace during execution of Python
>>>> script. You can see session example in attachment. The format is of PHP
>>>> Xdebug tool [2] just because I had some scripts from the past to analyze it.
>>>>
>>>> The module uses sys.settrace() to analyse frames in 'call' events with
>>>> callback(frame, event, arg).
>>>>
>>>> Recently I've got an anonymous report that some frame misses filename
>>>> information when run under IDE:
>>>>
>>>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>>>
>>>> There is unlikely to be any more feedback from the user to make me
>>>> understand and reproduce the behavior, nor there is a sufficient
>>>> documentation in sys.trace description [1]. So I am on my own. But I can
>>>> not read C code the CPython is written in, so I ask here.
>>>>
>>>> What are possible execution contexts for Python?
>>>> How each execution context affects data structure that is passed to
>>>> trace function (frame and arg)?
>>>>
>>>
>>>  It's simpler than that: when running from an interactive session,
>>> f_back is empty...
>>>
>>> >>> def trace(frame, event, arg):
>>> ...     funcname = frame.f_code.co_name
>>> ...     if funcname == '<module>':
>>> ...         print frame.f_back.f_code.co_filename
>>> ...
>>> >>> import sys
>>> >>> sys.settrace(trace)
>>> >>> 1
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>>   File "<stdin>", line 4, in trace
>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>
>>
>> Thanks a lot. =)
>>
>> User said he was using module from the IDE and at first I thought it
>> wasn't possible to use module without previous stack frame (f_back), but I
>> was wrong:
>>
>> > py -2 -s -S
>> Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)]
>> on win32
>> >>> import xtrace
>> >>> xtrace.start()
>> TRACE START [2013-03-20 18:49:27]
>> >>> z = 0
>>   -> decode() C:\Python27\lib\encodings\cp437.py:14
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "xtrace.py", line 66, in function_trace_xdebug
>>     filename = self._strip_cwd(frame.f_back.f_code.co_filename)
>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>
>>
>> On the way I've discovered that my Python is hacked. Without -s -S key it
>> executes modules from local temp directory.
>> ...
>>           -> ensure_unicode()
>> c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
>> dline\unicode_helper.py:20
>> ...
>>
>>
>> So there are absolutely no differences in running code in these execution
>> contexts?
>> - code runs from a file
>> - code runs from a interactive console
>>
>
> Of course there is a difference; the code below works run from a file, but
> not when pasted into the interactive console:
>
> def trace(frame, event, arg):
>     funcname = frame.f_code.co_name
>     print frame.f_code, repr(frame.f_code.co_code), event, arg
>     if funcname == '<module>':
>         print "    module", repr(frame.f_back.f_code.co_code)
>
> import sys
> sys.settrace(trace)
> exec '1 + 1'
>

For console it pretty clear why no previous frame exists - stack is
resolved at >>> prompt. But what is this top frame that appears inside file
context? The following code gives:
<module> 7
<module> 1

def trace(frame, event, arg):
    print frame.f_back.f_code.co_name, frame.f_back.f_lineno
    print frame.f_code.co_name, frame.f_lineno

import sys
sys.settrace(trace)
exec '1 + 1'

How to identify the frames here? How come that 7th line of module is
executed before 1 st? I expected to have module name available somewhere
for <module> object. All right, this kind of tricky - '1 +1' a separate
module, right? It is anonymous and doesn't have a name. Still is it
possible to identify it somehow? For printing in function traces.


>
>> I realize that I don't know how to set trace function for external file,
>> so that when the file is executed, the first frame won't have the previous
>> stack frame. I do it like so:
>>     start()
>>     execfile(script)
>>     stop()
>>
>
> I'ts indeed difficult to detach the "script" from the frame running the
> execfile statement.
> You could handle this case specially in your trace function, though.
>

Now that I understand, I can handle it. I guess stripping the 0 level calls
(where it will be only one) will help. Thanks again.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/845bf27c/attachment-0001.html>

From amauryfa at gmail.com  Wed Mar 20 18:21:54 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 20 Mar 2013 18:21:54 +0100
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAPkN8xJvc08rVnEzuj7XCbR20uvk4wx=M1Qy+UuwidpC-ZZWPg@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
	<CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
	<CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>
	<CAPkN8xJvc08rVnEzuj7XCbR20uvk4wx=M1Qy+UuwidpC-ZZWPg@mail.gmail.com>
Message-ID: <CAGmFidaQ2ja0rd=Ze7Lx03uQ7UofPXWDt5j0_4QqEpC81xSheQ@mail.gmail.com>

2013/3/20 anatoly techtonik <techtonik at gmail.com>

> On Wed, Mar 20, 2013 at 7:36 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:
>
>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>
>>> On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <
>>> amauryfa at gmail.com> wrote:
>>>
>>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>>
>>>>> Hi,
>>>>>
>>>>> I've created a module to dump function trace during execution of
>>>>> Python script. You can see session example in attachment. The format is of
>>>>> PHP Xdebug tool [2] just because I had some scripts from the past to
>>>>> analyze it.
>>>>>
>>>>> The module uses sys.settrace() to analyse frames in 'call' events with
>>>>> callback(frame, event, arg).
>>>>>
>>>>> Recently I've got an anonymous report that some frame misses filename
>>>>> information when run under IDE:
>>>>>
>>>>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>>>>
>>>>> There is unlikely to be any more feedback from the user to make me
>>>>> understand and reproduce the behavior, nor there is a sufficient
>>>>> documentation in sys.trace description [1]. So I am on my own. But I can
>>>>> not read C code the CPython is written in, so I ask here.
>>>>>
>>>>> What are possible execution contexts for Python?
>>>>> How each execution context affects data structure that is passed to
>>>>> trace function (frame and arg)?
>>>>>
>>>>
>>>>  It's simpler than that: when running from an interactive session,
>>>> f_back is empty...
>>>>
>>>> >>> def trace(frame, event, arg):
>>>> ...     funcname = frame.f_code.co_name
>>>> ...     if funcname == '<module>':
>>>> ...         print frame.f_back.f_code.co_filename
>>>> ...
>>>> >>> import sys
>>>> >>> sys.settrace(trace)
>>>> >>> 1
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>   File "<stdin>", line 4, in trace
>>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>>
>>>
>>> Thanks a lot. =)
>>>
>>> User said he was using module from the IDE and at first I thought it
>>> wasn't possible to use module without previous stack frame (f_back), but I
>>> was wrong:
>>>
>>> > py -2 -s -S
>>> Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
>>> (Intel)] on win32
>>> >>> import xtrace
>>> >>> xtrace.start()
>>> TRACE START [2013-03-20 18:49:27]
>>> >>> z = 0
>>>   -> decode() C:\Python27\lib\encodings\cp437.py:14
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>>   File "xtrace.py", line 66, in function_trace_xdebug
>>>     filename = self._strip_cwd(frame.f_back.f_code.co_filename)
>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>
>>>
>>> On the way I've discovered that my Python is hacked. Without -s -S key
>>> it executes modules from local temp directory.
>>> ...
>>>           -> ensure_unicode()
>>> c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
>>> dline\unicode_helper.py:20
>>> ...
>>>
>>>
>>> So there are absolutely no differences in running code in these
>>> execution contexts?
>>> - code runs from a file
>>> - code runs from a interactive console
>>>
>>
>> Of course there is a difference; the code below works run from a file,
>> but not when pasted into the interactive console:
>>
>> def trace(frame, event, arg):
>>     funcname = frame.f_code.co_name
>>     print frame.f_code, repr(frame.f_code.co_code), event, arg
>>     if funcname == '<module>':
>>         print "    module", repr(frame.f_back.f_code.co_code)
>>
>> import sys
>> sys.settrace(trace)
>> exec '1 + 1'
>>
>
> For console it pretty clear why no previous frame exists - stack is
> resolved at >>> prompt. But what is this top frame that appears inside file
> context? The following code gives:
> <module> 7
> <module> 1
>
> def trace(frame, event, arg):
>     print frame.f_back.f_code.co_name, frame.f_back.f_lineno
>     print frame.f_code.co_name, frame.f_lineno
>
> import sys
> sys.settrace(trace)
> exec '1 + 1'
>
> How to identify the frames here? How come that 7th line of module is
> executed before 1 st? I expected to have module name available somewhere
> for <module> object. All right, this kind of tricky - '1 +1' a separate
> module, right? It is anonymous and doesn't have a name. Still is it
> possible to identify it somehow? For printing in function traces.
>

An exception traceback gives you the answer; with the script foo.py below:

def f():
  exec 'x + 1'
f()

I get:

Traceback (most recent call last):
  File "/tmp/foo.py", line 4, in <module>
    f()
  File "/tmp/foo.py", line 2, in f
    exec 'x + 1'
  File "<string>", line 1, in <module>
NameError: name 'x' is not defined

Do you see all the information you need here?
If yes, it's possible to extract it from the current frames.


>
>>
>>> I realize that I don't know how to set trace function for external file,
>>> so that when the file is executed, the first frame won't have the previous
>>> stack frame. I do it like so:
>>>     start()
>>>     execfile(script)
>>>     stop()
>>>
>>
>> I'ts indeed difficult to detach the "script" from the frame running the
>> execfile statement.
>> You could handle this case specially in your trace function, though.
>>
>
> Now that I understand, I can handle it. I guess stripping the 0 level
> calls (where it will be only one) will help. Thanks again.
>



-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130320/9af97a53/attachment.html>

From matti.picus at gmail.com  Thu Mar 21 17:01:46 2013
From: matti.picus at gmail.com (matti picus)
Date: Thu, 21 Mar 2013 09:01:46 -0700
Subject: [pypy-dev] why doesn't buildbot master sort (after I told it to)
Message-ID: <CAO8bCgiwGoA7=djj7mO6dimkCxS4OCjvY+C76Wd_-MqT+DCttA@mail.gmail.com>

I know this is not all that important, but...
It annoyed me that there is so much stuff on this page
http://buildbot.pypy.org/nightly
so I changed our buildbot code to sort by filesystem mtime, and put
trunk on top.
I tried it out by writing tests, and even installing a debug buildbot,
and creating some directories in my local ~/nightly

It works locally but not on buildbot.
Any ideas why?
Matti

From joehillen at gmail.com  Thu Mar 21 18:10:12 2013
From: joehillen at gmail.com (Joe Hillenbrand)
Date: Thu, 21 Mar 2013 10:10:12 -0700
Subject: [pypy-dev] PyCon videos are up
Message-ID: <CANJQ5C4Ezs=ScqdME2KGDZeGc3N7v-YYMxLiXr0EXzhGrRY6Pg@mail.gmail.com>

For those on the list who have not seen Armin and Maciej's talk yet:

http://www.pyvideo.org/video/1739/pypy-without-the-gil

-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130321/c2e1199e/attachment.html>

From techtonik at gmail.com  Thu Mar 21 20:23:03 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 21 Mar 2013 22:23:03 +0300
Subject: [pypy-dev] frame.f_trace role
Message-ID: <CAPkN8xLeyP0YWSRRL6ZRdBmBz2Qaqnyt5uFjCvWGL89chYFdwQ@mail.gmail.com>

Hi,

I am puzzled. The documentation mentions writable .f_trace attribute for a
frame.
"if not None, is a function called at the start of each source code line"
http://docs.python.org/2/reference/datamodel.html#types

But there is also a trace function set by sys.settrace with 'line' event
http://docs.python.org/2/library/sys.html#sys.settrace

What's the difference? Why the f_trace is needed?

It is also unclear who sets f_trace, because it should be set at the
beginning of each frame before the frame starts executing.


Why do I need it? I wanted to make a scroller for a live Python code long
ago, and now that I am deep inside Python frames (thanks to xtrace
bugreport I got earlier) it sounds like a good time for it.

The only concern I have is if 'line' event will be enough to cover all the
code that Python executes in settrace set function.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130321/37f6db43/attachment.html>

From amauryfa at gmail.com  Thu Mar 21 20:49:30 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 21 Mar 2013 20:49:30 +0100
Subject: [pypy-dev] frame.f_trace role
In-Reply-To: <CAPkN8xLeyP0YWSRRL6ZRdBmBz2Qaqnyt5uFjCvWGL89chYFdwQ@mail.gmail.com>
References: <CAPkN8xLeyP0YWSRRL6ZRdBmBz2Qaqnyt5uFjCvWGL89chYFdwQ@mail.gmail.com>
Message-ID: <CAGmFidZzPGK+CZX48qbCp3-AEJn59jXaVR+aQ_nyW98sPWbCVw@mail.gmail.com>

Hi Anatoly,

You should really post this on python-users.
For these matters pypy mostly tries to re-implement CPython behavior,
without any consideration for a consistent design.

2013/3/21 anatoly techtonik <techtonik at gmail.com>

> Hi,
>
> I am puzzled. The documentation mentions writable .f_trace attribute for a
> frame.
> "if not None, is a function called at the start of each source code line"
> http://docs.python.org/2/reference/datamodel.html#types
>
> But there is also a trace function set by sys.settrace with 'line' event
> http://docs.python.org/2/library/sys.html#sys.settrace
>
> What's the difference? Why the f_trace is needed?
>
> It is also unclear who sets f_trace, because it should be set at the
> beginning of each frame before the frame starts executing.
>

A hint though: the docs for settrace say:
"""
'call'
A function is called (or some other code block entered). The global trace
function is called; arg is None; the return value specifies the local trace
function.
"""

So f_trace is the local trace function, sys.settrace is the global one.


> Why do I need it? I wanted to make a scroller for a live Python code long
> ago, and now that I am deep inside Python frames (thanks to xtrace
> bugreport I got earlier) it sounds like a good time for it.
>
> The only concern I have is if 'line' event will be enough to cover all the
> code that Python executes in settrace set function.
>

Profilers and code coverage tools use the trace function, so your case
should be covered as well.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130321/a774e9e5/attachment.html>

From anto.cuni at gmail.com  Thu Mar 21 21:20:35 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 21 Mar 2013 21:20:35 +0100
Subject: [pypy-dev] why doesn't buildbot master sort (after I told it to)
In-Reply-To: <CAO8bCgiwGoA7=djj7mO6dimkCxS4OCjvY+C76Wd_-MqT+DCttA@mail.gmail.com>
References: <CAO8bCgiwGoA7=djj7mO6dimkCxS4OCjvY+C76Wd_-MqT+DCttA@mail.gmail.com>
Message-ID: <514B6B93.2040100@gmail.com>

On 03/21/2013 05:01 PM, matti picus wrote:
> I know this is not all that important, but...
> It annoyed me that there is so much stuff on this page
> http://buildbot.pypy.org/nightly
> so I changed our buildbot code to sort by filesystem mtime, and put
> trunk on top.
> I tried it out by writing tests, and even installing a debug buildbot,
> and creating some directories in my local ~/nightly
>
> It works locally but not on buildbot.
> Any ideas why?

did you restart the buildmaster on cobra?


From matti.picus at gmail.com  Thu Mar 21 23:38:18 2013
From: matti.picus at gmail.com (Matti Picus)
Date: Thu, 21 Mar 2013 15:38:18 -0700
Subject: [pypy-dev] why doesn't buildbot master sort (after I told it to)
In-Reply-To: <514B6B93.2040100@gmail.com>
References: <CAO8bCgiwGoA7=djj7mO6dimkCxS4OCjvY+C76Wd_-MqT+DCttA@mail.gmail.com>
	<514B6B93.2040100@gmail.com>
Message-ID: <514B8BDA.6050603@gmail.com>

fijal did restart the buildbot, it didn't help.
Matti

On 21/03/2013 1:20 PM, Antonio Cuni wrote:
> On 03/21/2013 05:01 PM, matti picus wrote:
>> I know this is not all that important, but...
>> It annoyed me that there is so much stuff on this page
>> http://buildbot.pypy.org/nightly
>> so I changed our buildbot code to sort by filesystem mtime, and put
>> trunk on top.
>> I tried it out by writing tests, and even installing a debug buildbot,
>> and creating some directories in my local ~/nightly
>>
>> It works locally but not on buildbot.
>> Any ideas why?
>
> did you restart the buildmaster on cobra?
>


From matti.picus at gmail.com  Fri Mar 22 04:00:55 2013
From: matti.picus at gmail.com (matti picus)
Date: Thu, 21 Mar 2013 20:00:55 -0700
Subject: [pypy-dev] strange difference between py3k and default on win32
	build
Message-ID: <CAO8bCgi1a-Ex-fZmzWnCf4pQ+17xyRB2ZXh2jtjVhqGQ3DLAEg@mail.gmail.com>

I tried kicking the new windows buildbot on py3k branch, and got a
linking error about missing
zlib in openssl
http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/897/steps/translate/logs/stdio
but I couldn't find any relevant diff between the two branches that
would cause that. Before I commit
a random fix to the problem I would like to understand why.
Anyone have any ideas?
Matti

From anto.cuni at gmail.com  Fri Mar 22 09:56:14 2013
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 22 Mar 2013 09:56:14 +0100
Subject: [pypy-dev] why doesn't buildbot master sort (after I told it to)
In-Reply-To: <514B8BDA.6050603@gmail.com>
References: <CAO8bCgiwGoA7=djj7mO6dimkCxS4OCjvY+C76Wd_-MqT+DCttA@mail.gmail.com>
	<514B6B93.2040100@gmail.com> <514B8BDA.6050603@gmail.com>
Message-ID: <514C1CAE.3050202@gmail.com>

On 03/21/2013 11:38 PM, Matti Picus wrote:
> fijal did restart the buildbot, it didn't help.

then I fear that the best is to do some good old debug with print statements 
and pdb.set_trace().
If you login to cobra, you can do the following:

$ ssh buildmaster at cobra.cs.uni-duesseldorf.de
$ cd pypy-buildbot
$ less README # :-)
$ cd master
$ make stop
$ make debug

the nice thing of "make debug" instead of "make start" is that it does not 
redirect stdout to a logfile, which means that you can use pdb and prints to 
understand what's going wrong.

Once you fixed, remember to commit/push your changes and to "make start"
the master again.

ciao,
Anto

From amauryfa at gmail.com  Fri Mar 22 10:00:01 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 22 Mar 2013 10:00:01 +0100
Subject: [pypy-dev] strange difference between py3k and default on win32
	build
In-Reply-To: <CAO8bCgi1a-Ex-fZmzWnCf4pQ+17xyRB2ZXh2jtjVhqGQ3DLAEg@mail.gmail.com>
References: <CAO8bCgi1a-Ex-fZmzWnCf4pQ+17xyRB2ZXh2jtjVhqGQ3DLAEg@mail.gmail.com>
Message-ID: <CAGmFidbN2vpREvx+JGKCkPiCT7+V-7mn+AmuEiV7cs6yFN3mCg@mail.gmail.com>

2013/3/22 matti picus <matti.picus at gmail.com>

> I tried kicking the new windows buildbot on py3k branch, and got a
> linking error about missing
> zlib in openssl
>
> http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/897/steps/translate/logs/stdio
> but I couldn't find any relevant diff between the two branches that
> would cause that. Before I commit
> a random fix to the problem I would like to understand why.
> Anyone have any ideas?
>

Which version of openssl is installed?
It's possible that something important changed since the 0.9.8 series.
(and why can't we see any openssl related options in the compile commands?)

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130322/f12805bd/attachment.html>

From svpcom at gmail.com  Fri Mar 22 12:27:39 2013
From: svpcom at gmail.com (Vasily Evseenko)
Date: Fri, 22 Mar 2013 15:27:39 +0400
Subject: [pypy-dev] Memory corruption on socket.gethostbyname_ex
Message-ID: <514C402B.9020006@gmail.com>

Hi,

There are some memory corruption while calling socket.gethostbyname_ex
from the thread while other thread executes JIT'ed code:
In such case socket.gethostbyname_ex returns random entries from
/etc/hosts instead of DNS lookup.

------
import sys
import socket
import threading

domain = 'google.com'

def lookup(x):
    res = socket.gethostbyname_ex(x)
    if res[0] != 'google.com':
        print 'BUG: resolve %s -> %s' % (x, res)
        sys.stdout.flush()

def test():
    # do some JIT
    for i in xrange(1000000000):
        pass

for i in range(100):
    th = threading.Thread(target=lookup, args=[domain])
    th.start()

th = threading.Thread(target=test)
th.start()

print 'startup done'
------


Affected versions: PyPy 1.8, 1.9, 2.0b1 on Linux   (Fedora 16, Debian
GNU/Linux 6.0)
---
Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013,
07:39:19)
[PyPy 2.0.0-beta1 with GCC 4.4.5] on linux2

Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Dec 18 2012,
05:33:54)
[PyPy 2.0.0-beta1 with GCC 4.6.3] on linux2

Python 2.7.2 (2346207d99463f299f09f3e151c9d5fa9158f71b, May 15 2012,
11:08:08)
[PyPy 1.8.0 with GCC 4.6.3] on linux2

Python 2.7.2 (341e1e3821fff77db3bb5cdb7a4851626298c44e, Oct 23 2012,
11:37:42)
[PyPy 1.9.0 with GCC 4.6.3] on linux2
---

Not affected: PyPy 2.0b1 on MacOSX
---
Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013,
07:52:27)
[PyPy 2.0.0-beta1] on darwin

Python 2.7.3 (7e4f0faa3d51, Nov 21 2012, 15:59:46)
[PyPy 2.0.0-beta1 with GCC 4.2.1] on darwin
---




From amauryfa at gmail.com  Fri Mar 22 12:47:11 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 22 Mar 2013 12:47:11 +0100
Subject: [pypy-dev] Memory corruption on socket.gethostbyname_ex
In-Reply-To: <514C402B.9020006@gmail.com>
References: <514C402B.9020006@gmail.com>
Message-ID: <CAGmFida32+KX+wWRFdythYC9rSzC=StCXSKdTZpQWPxO-ek6Yg@mail.gmail.com>

Hi,

2013/3/22 Vasily Evseenko <svpcom at gmail.com>

> Hi,
>
> There are some memory corruption while calling socket.gethostbyname_ex
> from the thread while other thread executes JIT'ed code:
> In such case socket.gethostbyname_ex returns random entries from
> /etc/hosts instead of DNS lookup.
>

Please file a bug on https://bugs.pypy.org/
Some notes:
- it reproduces even without a JIT-enabled pypy
- PyPy uses gethostbyname() instead of gethostbyname_r()



>
> ------
> import sys
> import socket
> import threading
>
> domain = 'google.com'
>
> def lookup(x):
>     res = socket.gethostbyname_ex(x)
>     if res[0] != 'google.com':
>         print 'BUG: resolve %s -> %s' % (x, res)
>         sys.stdout.flush()
>
> def test():
>     # do some JIT
>     for i in xrange(1000000000):
>         pass
>
> for i in range(100):
>     th = threading.Thread(target=lookup, args=[domain])
>     th.start()
>
> th = threading.Thread(target=test)
> th.start()
>
> print 'startup done'
> ------
>
>
> Affected versions: PyPy 1.8, 1.9, 2.0b1 on Linux   (Fedora 16, Debian
> GNU/Linux 6.0)
> ---
> Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013,
> 07:39:19)
> [PyPy 2.0.0-beta1 with GCC 4.4.5] on linux2
>
> Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Dec 18 2012,
> 05:33:54)
> [PyPy 2.0.0-beta1 with GCC 4.6.3] on linux2
>
> Python 2.7.2 (2346207d99463f299f09f3e151c9d5fa9158f71b, May 15 2012,
> 11:08:08)
> [PyPy 1.8.0 with GCC 4.6.3] on linux2
>
> Python 2.7.2 (341e1e3821fff77db3bb5cdb7a4851626298c44e, Oct 23 2012,
> 11:37:42)
> [PyPy 1.9.0 with GCC 4.6.3] on linux2
> ---
>
> Not affected: PyPy 2.0b1 on MacOSX
> ---
> Python 2.7.3 (07e08e9c885ca67d89bcc304e45a32346daea2fa, Mar 19 2013,
> 07:52:27)
> [PyPy 2.0.0-beta1] on darwin
>
> Python 2.7.3 (7e4f0faa3d51, Nov 21 2012, 15:59:46)
> [PyPy 2.0.0-beta1 with GCC 4.2.1] on darwin
> ---
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130322/268cf3fa/attachment.html>

From mtasic85 at gmail.com  Sat Mar 23 21:07:21 2013
From: mtasic85 at gmail.com (Marko Tasic)
Date: Sat, 23 Mar 2013 21:07:21 +0100
Subject: [pypy-dev] Experimental GIL
Message-ID: <CAE3+++QL3XTULFFqNfq5ORdi7Ffsw84b_xs+usoSgYG_Q9rQ4Q@mail.gmail.com>

Hi,

I have had an idea for some time to re-implement how GIL works, but
nothing radical. On purpose I'm saying that I don't want to "remove"
GIL, but to modify it, so people who follow your list don't get too
excited.

Let me describe what I want to accomplish, and what kind of results am
looking for, and also hear any possible design or implementation issue
I/we may face.

I want threads to work in parallel as they already do, with very
little waiting times on other threads in case of GIL'ing. Actually, I
want to use time reserved for GIL to do something else.

To be clear, I don't need JIT, and lets take it out of discussion for
some time later.

First step:
-------------
My solution require wait-free lock-free queue, so many threads can
only append/push to it atomically. Every thread created, including
main thread, has to have (at least one) such a queue, and lets say
queue has pointer to thread where it belongs.

Not only queue is required, but also conditional variable (such as
pthread_cond_t) per thread.

What I imagine is high-level thread object that consists of low-level
thread, low-level wait-free lock-free queue, and low-level conditional
variable. Imagine that this high-level thread object is local per each
OS thread created. From now on I will call high-level thread object
just "thread". Every mutable object created has to have pointer to a
thread in which its created. As a result, this way we can know origin
of a mutable object by thread by inspecting mutable object itself, so
as a consequence also know low-level thread, low-level queue and
low-level condition variable associated by it.

Having this special field is naive approach, so of course we can have
allocation areas per thread, and GC can be smarter, but lets just
forget about it for now.

Second step:
-----------------
Every time "ceval" equivalent function observes a mutable object on
bytecode interpretation, it checks if local thread is same as object's
thread.

If they are same, then proceed with interpretation of local ceval operation.

If they are different, then:
    - lets call local thread just T
    - lets call operation just O
    - take low-level queue of object's thread, lets call it Q
    - group O and T into item and push it to Q
    - set T's condition variable to wait

Third step:
-------------
Lets imagine that we have old style GIL that counts opcodes executed,
and after N opcodes instead of acquiring GIL, it processes queue. This
processing is done by popping out operation (O) and thread (T) items
from local thread's queue, and serially executing operations. After
each executed operation it sends signal to condition variable passed
inside thread which was passed in same queue item where is operation.
Once, signal is received by awaiting thread, until now awaiting thread
proceeds with execution, while thread that was processing queue also
continues executing queued operations.

Conclusion:
---------------
There is ceval per thread. All ceval's run at same time. In the best
case, they don't wait on each other because threads do not touch
objects created in other threads. In the worst case, there is always
only one thread executing operation, and all other are waiting it to
finish, and this is close to how GIL already works (almost).

Special care has to be taken for threads that are going to finish, and
what happens with their objects if they are referenced by other
objects that belong to other threads.

So I have also some questions:
- How flexible is to modify GIL in pypy?
- Where should I look?
- Do I need to fully translate pypy in order to test it?
- I need wait-free look-free queue, so what kind of low-level atomic
primitives do RPython has to offer?
- I need to have special field for all mutable objects, so how to do so?

I do not expect any performance boost, just correctness of approach.
Perhaps, I'm wrong.

What do you think, is this practical approach?

Regards,
Marko Tasic

From cfbolz at gmx.de  Sat Mar 23 21:32:43 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 23 Mar 2013 21:32:43 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: (fijal,
	alex) kill 'int' as a multimethod
In-Reply-To: <20130323190209.6105D1C016E@cobra.cs.uni-duesseldorf.de>
References: <20130323190209.6105D1C016E@cobra.cs.uni-duesseldorf.de>
Message-ID: <514E116B.9080205@gmx.de>

Hi Maciej, Alex,

I really don't like this commit. It does two unrelated pretty
fundamental changes together, killing small integers and making int a
normal method. The first of these is not even explained in the commit
message.

I understand the reasons for removing small integers (they don't work
with the JIT anyway, fixing that would be hard, ...). But it would
still be good to have two commits, one that removes them, the other one
that does your refactoring. This would give people that want to revive
them at a later point a place to start.

I don't actually know what should be done now. Backing out your change
and doing it in two steps would be the right approach somehow, even if
it requires some work.

Cheers,

Carl Friedrich


On 03/23/2013 08:02 PM, fijal wrote:
> Author: Maciej Fijalkowski <fijall at gmail.com>
> Branch:
> Changeset: r62700:e258f1470515
> Date: 2013-03-23 12:01 -0700
> http://bitbucket.org/pypy/pypy/changeset/e258f1470515/
>
> Log:	(fijal, alex) kill 'int' as a multimethod
>
> diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
> --- a/pypy/config/pypyoption.py
> +++ b/pypy/config/pypyoption.py
> @@ -189,11 +189,6 @@
>           BoolOption("withtproxy", "support transparent proxies",
>                      default=True),
>
> -        BoolOption("withsmallint", "use tagged integers",
> -                   default=False,
> -                   requires=[("objspace.std.withprebuiltint", False),
> -                             ("translation.taggedpointers", True)]),
> -
>           BoolOption("withprebuiltint", "prebuild commonly used int objects",
>                      default=False),
>
> @@ -204,9 +199,7 @@
>                     default=100, cmdline="--prebuiltintto"),
>
>           BoolOption("withsmalllong", "use a version of 'long' in a C long long",
> -                   default=False,
> -                   requires=[("objspace.std.withsmallint", False)]),
> -                             #  ^^^ because of missing delegate_xx2yy
> +                   default=False),
>
>           BoolOption("withstrbuf", "use strings optimized for addition (ver 2)",
>                      default=False),
> diff --git a/pypy/interpreter/baseobjspace.py b/pypy/interpreter/baseobjspace.py
> --- a/pypy/interpreter/baseobjspace.py
> +++ b/pypy/interpreter/baseobjspace.py
> @@ -223,6 +223,22 @@
>           raise OperationError(space.w_TypeError,
>                                typed_unwrap_error_msg(space, "integer", self))
>
> +    def int(self, space):
> +        w_impl = space.lookup(self, '__int__')
> +        if w_impl is None:
> +            typename = space.type(self).getname(space)
> +            raise operationerrfmt(space.w_TypeError,
> +                  "unsupported operand type for int(): '%s'",
> +                                  typename)
> +        w_result = space.get_and_call_function(w_impl, self)
> +
> +        if (space.isinstance_w(w_result, space.w_int) or
> +            space.isinstance_w(w_result, space.w_long)):
> +            return w_result
> +        typename = space.type(w_result).getname(space)
> +        msg = "__int__ returned non-int (type '%s')"
> +        raise operationerrfmt(space.w_TypeError, msg, typename)
> +
>       def __spacebind__(self, space):
>           return self
>
> @@ -1306,6 +1322,9 @@
>       def int_w(self, w_obj):
>           return w_obj.int_w(self)
>
> +    def int(self, w_obj):
> +        return w_obj.int(self)
> +
>       def uint_w(self, w_obj):
>           return w_obj.uint_w(self)
>
> diff --git a/pypy/objspace/descroperation.py b/pypy/objspace/descroperation.py
> --- a/pypy/objspace/descroperation.py
> +++ b/pypy/objspace/descroperation.py
> @@ -808,7 +808,6 @@
>   # more of the above manually-coded operations as well)
>
>   for targetname, specialname, checkerspec in [
> -    ('int', '__int__', ("space.w_int", "space.w_long")),
>       ('index', '__index__', ("space.w_int", "space.w_long")),
>       ('long', '__long__', ("space.w_int", "space.w_long")),
>       ('float', '__float__', ("space.w_float",))]:
> diff --git a/pypy/objspace/std/complexobject.py b/pypy/objspace/std/complexobject.py
> --- a/pypy/objspace/std/complexobject.py
> +++ b/pypy/objspace/std/complexobject.py
> @@ -111,6 +111,9 @@
>
>           return w_result
>
> +    def int(self, space):
> +        raise OperationError(space.w_TypeError, space.wrap("can't convert complex to int; use int(abs(z))"))
> +
>   registerimplementation(W_ComplexObject)
>
>   w_one = W_ComplexObject(1, 0)
> @@ -245,9 +248,6 @@
>   def float__Complex(space, w_complex):
>       raise OperationError(space.w_TypeError, space.wrap("can't convert complex to float; use abs(z)"))
>
> -def int__Complex(space, w_complex):
> -    raise OperationError(space.w_TypeError, space.wrap("can't convert complex to int; use int(abs(z))"))
> -
>   def complex_conjugate__Complex(space, w_self):
>       #w_real = space.call_function(space.w_float,space.wrap(w_self.realval))
>       #w_imag = space.call_function(space.w_float,space.wrap(-w_self.imagval))
> diff --git a/pypy/objspace/std/floatobject.py b/pypy/objspace/std/floatobject.py
> --- a/pypy/objspace/std/floatobject.py
> +++ b/pypy/objspace/std/floatobject.py
> @@ -58,6 +58,14 @@
>       def float_w(self, space):
>           return self.floatval
>
> +    def int(self, space):
> +        try:
> +            value = ovfcheck_float_to_int(self.floatval)
> +        except OverflowError:
> +            return space.long(self)
> +        else:
> +            return space.newint(value)
> +
>       def __repr__(self):
>           return "<W_FloatObject(%f)>" % self.floatval
>
> @@ -89,14 +97,6 @@
>       a = w_float1.floatval
>       return W_FloatObject(a)
>
> -def int__Float(space, w_value):
> -    try:
> -        value = ovfcheck_float_to_int(w_value.floatval)
> -    except OverflowError:
> -        return space.long(w_value)
> -    else:
> -        return space.newint(value)
> -
>   def long__Float(space, w_floatobj):
>       try:
>           return W_LongObject.fromfloat(space, w_floatobj.floatval)
> diff --git a/pypy/objspace/std/frame.py b/pypy/objspace/std/frame.py
> --- a/pypy/objspace/std/frame.py
> +++ b/pypy/objspace/std/frame.py
> @@ -6,7 +6,7 @@
>   from pypy.interpreter import pyopcode
>   from pypy.interpreter.pyframe import PyFrame
>   from pypy.interpreter.error import OperationError
> -from pypy.objspace.std import intobject, smallintobject
> +from pypy.objspace.std import intobject
>   from pypy.objspace.std.multimethod import FailedToImplement
>   from pypy.objspace.std.listobject import W_ListObject
>
> @@ -23,20 +23,6 @@
>               raise AssertionError
>
>
> -def small_int_BINARY_ADD(f, oparg, next_instr):
> -    w_2 = f.popvalue()
> -    w_1 = f.popvalue()
> -    if (type(w_1) is smallintobject.W_SmallIntObject and
> -        type(w_2) is smallintobject.W_SmallIntObject):
> -        try:
> -            w_result = smallintobject.add__SmallInt_SmallInt(f.space, w_1, w_2)
> -        except FailedToImplement:
> -            w_result = f.space.add(w_1, w_2)
> -    else:
> -        w_result = f.space.add(w_1, w_2)
> -    f.pushvalue(w_result)
> -
> -
>   def int_BINARY_ADD(f, oparg, next_instr):
>       w_2 = f.popvalue()
>       w_1 = f.popvalue()
> @@ -102,10 +88,7 @@
>       class StdObjSpaceFrame(BaseFrame):
>           pass
>       if space.config.objspace.std.optimized_int_add:
> -        if space.config.objspace.std.withsmallint:
> -            StdObjSpaceFrame.BINARY_ADD = small_int_BINARY_ADD
> -        else:
> -            StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD
> +        StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD
>       if space.config.objspace.std.optimized_list_getitem:
>           StdObjSpaceFrame.BINARY_SUBSCR = list_BINARY_SUBSCR
>       if space.config.objspace.opcodes.CALL_METHOD:
> diff --git a/pypy/objspace/std/intobject.py b/pypy/objspace/std/intobject.py
> --- a/pypy/objspace/std/intobject.py
> +++ b/pypy/objspace/std/intobject.py
> @@ -1,6 +1,6 @@
>   from pypy.interpreter.error import OperationError
>   from pypy.objspace.std import newformat
> -from pypy.objspace.std.inttype import wrapint
> +from pypy.objspace.std.inttype import wrapint, W_AbstractIntObject
>   from pypy.objspace.std.model import registerimplementation, W_Object
>   from pypy.objspace.std.multimethod import FailedToImplementArgs
>   from pypy.objspace.std.noneobject import W_NoneObject
> @@ -16,25 +16,6 @@
>   something CPython does not do anymore.
>   """
>
> -class W_AbstractIntObject(W_Object):
> -    __slots__ = ()
> -
> -    def is_w(self, space, w_other):
> -        if not isinstance(w_other, W_AbstractIntObject):
> -            return False
> -        if self.user_overridden_class or w_other.user_overridden_class:
> -            return self is w_other
> -        return space.int_w(self) == space.int_w(w_other)
> -
> -    def immutable_unique_id(self, space):
> -        if self.user_overridden_class:
> -            return None
> -        from pypy.objspace.std.model import IDTAG_INT as tag
> -        b = space.bigint_w(self)
> -        b = b.lshift(3).or_(rbigint.fromint(tag))
> -        return space.newlong_from_rbigint(b)
> -
> -
>   class W_IntObject(W_AbstractIntObject):
>       __slots__ = 'intval'
>       _immutable_fields_ = ['intval']
> @@ -67,6 +48,17 @@
>       def float_w(self, space):
>           return float(self.intval)
>
> +    def int(self, space):
> +        # XXX find a better way to do it
> +        if (type(self) != W_IntObject and
> +            space.lookup(self, '__int__') is not
> +            space.lookup_in_type_where(space.w_int, '__int__')[1]):
> +            return W_Object.int(self, space)
> +        if space.is_w(space.type(self), space.w_int):
> +            return self
> +        a = self.intval
> +        return wrapint(space, a)
> +
>   registerimplementation(W_IntObject)
>
>   # NB: This code is shared by smallintobject.py, and thus no other Int
> @@ -104,7 +96,7 @@
>       # unlike CPython, we don't special-case the value -1 in most of our
>       # hash functions, so there is not much sense special-casing it here either.
>       # Make sure this is consistent with the hash of floats and longs.
> -    return get_integer(space, w_int1)
> +    return w_int1.int(space)
>
>   # coerce
>   def coerce__Int_Int(space, w_int1, w_int2):
> @@ -251,7 +243,7 @@
>
>   def abs__Int(space, w_int1):
>       if w_int1.intval >= 0:
> -        return get_integer(space, w_int1)
> +        return w_int1.int(space)
>       else:
>           return get_negint(space, w_int1)
>
> @@ -278,7 +270,7 @@
>                                space.wrap("negative shift count"))
>       else: #b >= LONG_BIT
>           if a == 0:
> -            return get_integer(space, w_int1)
> +            return w_int1.int(space)
>           raise FailedToImplementArgs(space.w_OverflowError,
>                                   space.wrap("integer left shift"))
>
> @@ -291,7 +283,7 @@
>                                    space.wrap("negative shift count"))
>           else: # b >= LONG_BIT
>               if a == 0:
> -                return get_integer(space, w_int1)
> +                return w_int1.int(space)
>               if a < 0:
>                   a = -1
>               else:
> @@ -321,17 +313,13 @@
>   # int__Int is supposed to do nothing, unless it has
>   # a derived integer object, where it should return
>   # an exact one.
> -def int__Int(space, w_int1):
> -    if space.is_w(space.type(w_int1), space.w_int):
> -        return w_int1
> -    a = w_int1.intval
> -    return wrapint(space, a)
> -get_integer = int__Int
> -pos__Int = int__Int
> -trunc__Int = int__Int
> +
> +def pos__Int(self, space):
> +    return self.int(space)
> +trunc__Int = pos__Int
>
>   def index__Int(space, w_int1):
> -    return get_integer(space, w_int1)
> +    return w_int1.int(space)
>
>   def float__Int(space, w_int1):
>       a = w_int1.intval
> diff --git a/pypy/objspace/std/inttype.py b/pypy/objspace/std/inttype.py
> --- a/pypy/objspace/std/inttype.py
> +++ b/pypy/objspace/std/inttype.py
> @@ -1,5 +1,6 @@
>   from pypy.interpreter import typedef
> -from pypy.interpreter.gateway import interp2app, unwrap_spec, WrappedDefault
> +from pypy.interpreter.gateway import interp2app, unwrap_spec, WrappedDefault,\
> +     interpindirect2app
>   from pypy.interpreter.error import OperationError, operationerrfmt
>   from pypy.interpreter.buffer import Buffer
>   from pypy.objspace.std.register_all import register_all
> @@ -7,8 +8,10 @@
>   from pypy.objspace.std.strutil import (string_to_int, string_to_bigint,
>                                          ParseStringError,
>                                          ParseStringOverflowError)
> +from pypy.objspace.std.model import W_Object
>   from rpython.rlib.rarithmetic import r_uint
>   from rpython.rlib.objectmodel import instantiate
> +from rpython.rlib.rbigint import rbigint
>
>   # ____________________________________________________________
>
> @@ -36,14 +39,7 @@
>
>
>   def wrapint(space, x):
> -    if space.config.objspace.std.withsmallint:
> -        from pypy.objspace.std.smallintobject import W_SmallIntObject
> -        try:
> -            return W_SmallIntObject(x)
> -        except OverflowError:
> -            from pypy.objspace.std.intobject import W_IntObject
> -            return W_IntObject(x)
> -    elif space.config.objspace.std.withprebuiltint:
> +    if space.config.objspace.std.withprebuiltint:
>           from pypy.objspace.std.intobject import W_IntObject
>           lower = space.config.objspace.std.prebuiltintfrom
>           upper =  space.config.objspace.std.prebuiltintto
> @@ -185,6 +181,27 @@
>
>   # ____________________________________________________________
>
> +class W_AbstractIntObject(W_Object):
> +    __slots__ = ()
> +
> +    def is_w(self, space, w_other):
> +        if not isinstance(w_other, W_AbstractIntObject):
> +            return False
> +        if self.user_overridden_class or w_other.user_overridden_class:
> +            return self is w_other
> +        return space.int_w(self) == space.int_w(w_other)
> +
> +    def immutable_unique_id(self, space):
> +        if self.user_overridden_class:
> +            return None
> +        from pypy.objspace.std.model import IDTAG_INT as tag
> +        b = space.bigint_w(self)
> +        b = b.lshift(3).or_(rbigint.fromint(tag))
> +        return space.newlong_from_rbigint(b)
> +
> +    def int(self, space):
> +        raise NotImplementedError
> +
>   int_typedef = StdTypeDef("int",
>       __doc__ = '''int(x[, base]) -> integer
>
> @@ -201,5 +218,6 @@
>       denominator = typedef.GetSetProperty(descr_get_denominator),
>       real = typedef.GetSetProperty(descr_get_real),
>       imag = typedef.GetSetProperty(descr_get_imag),
> +    __int__ = interpindirect2app(W_AbstractIntObject.int),
>   )
>   int_typedef.registermethods(globals())
> diff --git a/pypy/objspace/std/longobject.py b/pypy/objspace/std/longobject.py
> --- a/pypy/objspace/std/longobject.py
> +++ b/pypy/objspace/std/longobject.py
> @@ -87,6 +87,12 @@
>       def float_w(self, space):
>           return self.num.tofloat()
>
> +    def int(self, space):
> +        try:
> +            return space.newint(self.num.toint())
> +        except OverflowError:
> +            return long__Long(space, self)
> +
>       def __repr__(self):
>           return '<W_LongObject(%d)>' % self.num.tolong()
>
> @@ -130,12 +136,6 @@
>   def long__Int(space, w_intobj):
>       return space.newlong(w_intobj.intval)
>
> -def int__Long(space, w_value):
> -    try:
> -        return space.newint(w_value.num.toint())
> -    except OverflowError:
> -        return long__Long(space, w_value)
> -
>   def index__Long(space, w_value):
>       return long__Long(space, w_value)
>
> diff --git a/pypy/objspace/std/model.py b/pypy/objspace/std/model.py
> --- a/pypy/objspace/std/model.py
> +++ b/pypy/objspace/std/model.py
> @@ -17,7 +17,6 @@
>   option_to_typename = {
>       "withspecialisedtuple" : ["specialisedtupleobject.W_SpecialisedTupleObject"],
>       "withsmalltuple" : ["smalltupleobject.W_SmallTupleObject"],
> -    "withsmallint"   : ["smallintobject.W_SmallIntObject"],
>       "withsmalllong"  : ["smalllongobject.W_SmallLongObject"],
>       "withstrbuf"     : ["strbufobject.W_StringBufferObject"],
>   }
> @@ -158,18 +157,6 @@
>           # when trying to dispatch multimethods.
>           # XXX build these lists a bit more automatically later
>
> -        if config.objspace.std.withsmallint:
> -            from pypy.objspace.std import smallintobject
> -            self.typeorder[boolobject.W_BoolObject] += [
> -                (smallintobject.W_SmallIntObject, boolobject.delegate_Bool2SmallInt),
> -                ]
> -            self.typeorder[smallintobject.W_SmallIntObject] += [
> -                (intobject.W_IntObject, smallintobject.delegate_SmallInt2Int),
> -                (floatobject.W_FloatObject, smallintobject.delegate_SmallInt2Float),
> -                (longobject.W_LongObject, smallintobject.delegate_SmallInt2Long),
> -                (complexobject.W_ComplexObject, smallintobject.delegate_SmallInt2Complex),
> -                ]
> -
>           if config.objspace.usemodules.micronumpy:
>               from pypy.module.micronumpy.stdobjspace import register_delegates
>               register_delegates(self.typeorder)
> @@ -424,7 +411,7 @@
>       ['delattr', 'delete', 'get', 'id', 'inplace_div', 'inplace_floordiv',
>        'inplace_lshift', 'inplace_mod', 'inplace_pow', 'inplace_rshift',
>        'inplace_truediv', 'is_', 'set', 'setattr', 'type', 'userdel',
> -     'isinstance', 'issubtype'])
> +     'isinstance', 'issubtype', 'int'])
>   # XXX should we just remove those from the method table or we're happy
>   #     with just not having multimethods?
>
> @@ -442,7 +429,7 @@
>
>       # add all regular multimethods here
>       for _name, _symbol, _arity, _specialnames in ObjSpace.MethodTable:
> -        if _name not in locals() or _name in NOT_MULTIMETHODS:
> +        if _name not in locals() and _name not in NOT_MULTIMETHODS:
>               mm = StdObjSpaceMultiMethod(_symbol, _arity, _specialnames)
>               locals()[_name] = mm
>               del mm
> diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
> --- a/pypy/objspace/std/objspace.py
> +++ b/pypy/objspace/std/objspace.py
> @@ -26,7 +26,6 @@
>   from pypy.objspace.std.iterobject import W_SeqIterObject
>   from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
>   from pypy.objspace.std.sliceobject import W_SliceObject
> -from pypy.objspace.std.smallintobject import W_SmallIntObject
>   from pypy.objspace.std.stringobject import W_StringObject
>   from pypy.objspace.std.unicodeobject import W_UnicodeObject
>   from pypy.objspace.std.tupleobject import W_AbstractTupleObject
> @@ -580,16 +579,8 @@
>               self.setitem(w_obj, self.wrap(key), w_value)
>
>       def getindex_w(self, w_obj, w_exception, objdescr=None):
> -        # Performance shortcut for the common case of w_obj being an int.
> -        # If withsmallint is disabled, we check for W_IntObject.
> -        # If withsmallint is enabled, we only check for W_SmallIntObject - it's
> -        # probably not useful to have a shortcut for W_IntObject at all then.
> -        if self.config.objspace.std.withsmallint:
> -            if type(w_obj) is W_SmallIntObject:
> -                return w_obj.intval
> -        else:
> -            if type(w_obj) is W_IntObject:
> -                return w_obj.intval
> +        if type(w_obj) is W_IntObject:
> +            return w_obj.intval
>           return ObjSpace.getindex_w(self, w_obj, w_exception, objdescr)
>
>       def call_method(self, w_obj, methname, *arg_w):
> diff --git a/pypy/objspace/std/smallintobject.py b/pypy/objspace/std/smallintobject.py
> deleted file mode 100644
> --- a/pypy/objspace/std/smallintobject.py
> +++ /dev/null
> @@ -1,87 +0,0 @@
> -"""
> -Implementation of small ints, stored as odd-valued pointers in the
> -translated PyPy.  To enable them, see inttype.py.
> -"""
> -from pypy.objspace.std import intobject
> -from pypy.objspace.std.model import registerimplementation, W_Object
> -from pypy.objspace.std.register_all import register_all
> -from pypy.objspace.std.noneobject import W_NoneObject
> -from pypy.objspace.std.intobject import W_AbstractIntObject, W_IntObject
> -from pypy.interpreter.error import OperationError
> -from rpython.rlib.objectmodel import UnboxedValue
> -from rpython.rlib.rbigint import rbigint
> -from rpython.rlib.rarithmetic import r_uint
> -from rpython.tool.sourcetools import func_with_new_name
> -from pypy.objspace.std.inttype import wrapint
> -
> -class W_SmallIntObject(W_AbstractIntObject, UnboxedValue):
> -    __slots__ = 'intval'
> -    from pypy.objspace.std.inttype import int_typedef as typedef
> -
> -    def unwrap(w_self, space):
> -        return int(w_self.intval)
> -    int_w = unwrap
> -
> -    def uint_w(w_self, space):
> -        intval = w_self.intval
> -        if intval < 0:
> -            raise OperationError(space.w_ValueError,
> -                                 space.wrap("cannot convert negative integer to unsigned"))
> -        else:
> -            return r_uint(intval)
> -
> -    def bigint_w(w_self, space):
> -        return rbigint.fromint(w_self.intval)
> -
> -
> -registerimplementation(W_SmallIntObject)
> -
> -
> -def delegate_SmallInt2Int(space, w_small):
> -    return W_IntObject(w_small.intval)
> -
> -def delegate_SmallInt2Long(space, w_small):
> -    return space.newlong(w_small.intval)
> -
> -def delegate_SmallInt2Float(space, w_small):
> -    return space.newfloat(float(w_small.intval))
> -
> -def delegate_SmallInt2Complex(space, w_small):
> -    return space.newcomplex(float(w_small.intval), 0.0)
> -
> -def add__SmallInt_SmallInt(space, w_a, w_b):
> -    return wrapint(space, w_a.intval + w_b.intval) # cannot overflow
> -
> -def sub__SmallInt_SmallInt(space, w_a, w_b):
> -    return wrapint(space, w_a.intval - w_b.intval) # cannot overflow
> -
> -def floordiv__SmallInt_SmallInt(space, w_a, w_b):
> -    return wrapint(space, w_a.intval // w_b.intval) # cannot overflow
> -
> -div__SmallInt_SmallInt = floordiv__SmallInt_SmallInt
> -
> -def mod__SmallInt_SmallInt(space, w_a, w_b):
> -    return wrapint(space, w_a.intval % w_b.intval) # cannot overflow
> -
> -def divmod__SmallInt_SmallInt(space, w_a, w_b):
> -    w = wrapint(space, w_a.intval // w_b.intval) # cannot overflow
> -    z = wrapint(space, w_a.intval % w_b.intval)
> -    return space.newtuple([w, z])
> -
> -def copy_multimethods(ns):
> -    """Copy integer multimethods for small int."""
> -    for name, func in intobject.__dict__.iteritems():
> -        if "__Int" in name:
> -            new_name = name.replace("Int", "SmallInt")
> -            if new_name not in ns:
> -                # Copy the function, so the annotator specializes it for
> -                # W_SmallIntObject.
> -                ns[new_name] = func = func_with_new_name(func, new_name, globals=ns)
> -        else:
> -            ns[name] = func
> -    ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"]
> -    ns["get_negint"] = ns["neg__SmallInt"]
> -
> -copy_multimethods(globals())
> -
> -register_all(vars())
> diff --git a/pypy/objspace/std/smalllongobject.py b/pypy/objspace/std/smalllongobject.py
> --- a/pypy/objspace/std/smalllongobject.py
> +++ b/pypy/objspace/std/smalllongobject.py
> @@ -69,7 +69,15 @@
>       def float_w(self, space):
>           return float(self.longlong)
>
> +    def int(self, space):
> +        a = self.longlong
> +        b = intmask(a)
> +        if b == a:
> +            return space.newint(b)
> +        else:
> +            return self
>
> +
>   registerimplementation(W_SmallLongObject)
>
>   # ____________________________________________________________
> @@ -119,14 +127,6 @@
>   def long__SmallLong(space, w_value):
>       return w_value
>
> -def int__SmallLong(space, w_value):
> -    a = w_value.longlong
> -    b = intmask(a)
> -    if b == a:
> -        return space.newint(b)
> -    else:
> -        return w_value
> -
>   def index__SmallLong(space, w_value):
>       return w_value
>
> diff --git a/pypy/objspace/std/test/test_intobject.py b/pypy/objspace/std/test/test_intobject.py
> --- a/pypy/objspace/std/test/test_intobject.py
> +++ b/pypy/objspace/std/test/test_intobject.py
> @@ -268,7 +268,7 @@
>
>       def test_int(self):
>           f1 = iobj.W_IntObject(1)
> -        result = iobj.int__Int(self.space, f1)
> +        result = f1.int(self.space)
>           assert result == f1
>
>       def test_oct(self):
> diff --git a/pypy/objspace/std/test/test_smallintobject.py b/pypy/objspace/std/test/test_smallintobject.py
> deleted file mode 100644
> --- a/pypy/objspace/std/test/test_smallintobject.py
> +++ /dev/null
> @@ -1,229 +0,0 @@
> -import sys, py
> -
> -#from pypy.objspace.std.model import WITHSMALLINT
> -#if not WITHSMALLINT:
> -#    py.test.skip("WITHSMALLINT is not enabled")
> -
> -from pypy.objspace.std.inttype import wrapint
> -from pypy.objspace.std.multimethod import FailedToImplement
> -from rpython.rlib.rarithmetic import r_uint
> -
> -from pypy.objspace.std.test.test_intobject import AppTestInt
> -
> -class TestW_IntObject:
> -    spaceconfig = {"objspace.std.withsmallint": True}
> -
> -    def test_int_w(self):
> -        assert self.space.int_w(self.space.wrap(42)) == 42
> -
> -    def test_uint_w(self):
> -        space = self.space
> -        assert space.uint_w(space.wrap(42)) == 42
> -        assert isinstance(space.uint_w(space.wrap(42)), r_uint)
> -        space.raises_w(space.w_ValueError, space.uint_w, space.wrap(-1))
> -
> -    def test_repr(self):
> -        x = 1
> -        f1 = wrapint(self.space, x)
> -        result = self.space.repr(f1)
> -        assert self.space.unwrap(result) == repr(x)
> -
> -    def test_str(self):
> -        x = 12345
> -        f1 = wrapint(self.space, x)
> -        result = self.space.str(f1)
> -        assert self.space.unwrap(result) == str(x)
> -
> -    def test_hash(self):
> -        x = 42
> -        f1 = wrapint(self.space, x)
> -        result = self.space.hash(f1)
> -        assert result.intval == hash(x)
> -
> -    def test_compare(self):
> -        import operator
> -        optab = ['lt', 'le', 'eq', 'ne', 'gt', 'ge']
> -        for x in (-10, -1, 0, 1, 2, 1000, sys.maxint):
> -            for y in (-sys.maxint-1, -11, -9, -2, 0, 1, 3, 1111, sys.maxint):
> -                for op in optab:
> -                    wx = wrapint(self.space, x)
> -                    wy = wrapint(self.space, y)
> -                    res = getattr(operator, op)(x, y)
> -                    method = getattr(self.space, op)
> -                    myres = method(wx, wy)
> -                    assert self.space.unwrap(myres) == res
> -
> -    def test_add(self):
> -        for x in [1, 100, sys.maxint // 2 - 50,
> -                  sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
> -            for y in [1, 100, sys.maxint // 2 - 50,
> -                      sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
> -                f1 = wrapint(self.space, x)
> -                f2 = wrapint(self.space, y)
> -                result = self.space.unwrap(self.space.add(f1, f2))
> -                assert result == x+y
> -
> -    def test_sub(self):
> -        for x in [1, 100, sys.maxint // 2 - 50,
> -                  sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
> -            for y in [1, 100, sys.maxint // 2 - 50,
> -                      sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
> -                f1 = wrapint(self.space, x)
> -                f2 = wrapint(self.space, y)
> -                result = self.space.unwrap(self.space.sub(f1, f2))
> -                assert result == x-y
> -
> -    def test_mul(self):
> -        for x in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint - 1000]:
> -            for y in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint - 1000]:
> -                f1 = wrapint(self.space, x)
> -                f2 = wrapint(self.space, y)
> -                result = self.space.unwrap(self.space.mul(f1, f2))
> -                assert result == x*y
> -
> -
> -    def test_div(self):
> -        for i in range(10):
> -            res = i//3
> -            f1 = wrapint(self.space, i)
> -            f2 = wrapint(self.space, 3)
> -            result = self.space.div(f1, f2)
> -            assert result.intval == res
> -
> -    def test_mod(self):
> -        x = 1
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.mod(f1, f2)
> -        assert v.intval == x % y
> -        # not that mod cannot overflow
> -
> -    def test_divmod(self):
> -        x = 1
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        ret = self.space.divmod(f1, f2)
> -        v, w = self.space.unwrap(ret)
> -        assert (v, w) == divmod(x, y)
> -
> -    def test_pow_iii(self):
> -        x = 10
> -        y = 2
> -        z = 13
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        f3 = wrapint(self.space, z)
> -        v = self.space.pow(f1, f2, f3)
> -        assert v.intval == pow(x, y, z)
> -        f1, f2, f3 = [wrapint(self.space, i) for i in (10, -1, 42)]
> -        self.space.raises_w(self.space.w_TypeError,
> -                            self.space.pow,
> -                            f1, f2, f3)
> -        f1, f2, f3 = [wrapint(self.space, i) for i in (10, 5, 0)]
> -        self.space.raises_w(self.space.w_ValueError,
> -                            self.space.pow,
> -                            f1, f2, f3)
> -
> -    def test_pow_iin(self):
> -        x = 10
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.pow(f1, f2, self.space.w_None)
> -        assert v.intval == x ** y
> -
> -    def test_neg(self):
> -        x = 42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.neg(f1)
> -        assert v.intval == -x
> -
> -    def test_pos(self):
> -        x = 42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.pos(f1)
> -        assert v.intval == +x
> -        x = -42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.pos(f1)
> -        assert v.intval == +x
> -
> -    def test_abs(self):
> -        x = 42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.abs(f1)
> -        assert v.intval == abs(x)
> -        x = -42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.abs(f1)
> -        assert v.intval == abs(x)
> -
> -    def test_invert(self):
> -        x = 42
> -        f1 = wrapint(self.space, x)
> -        v = self.space.invert(f1)
> -        assert v.intval == ~x
> -
> -    def test_lshift(self):
> -        x = 12345678
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.lshift(f1, f2)
> -        assert v.intval == x << y
> -
> -    def test_rshift(self):
> -        x = 12345678
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.rshift(f1, f2)
> -        assert v.intval == x >> y
> -
> -    def test_and(self):
> -        x = 12345678
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.and_(f1, f2)
> -        assert v.intval == x & y
> -
> -    def test_xor(self):
> -        x = 12345678
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.xor(f1, f2)
> -        assert v.intval == x ^ y
> -
> -    def test_or(self):
> -        x = 12345678
> -        y = 2
> -        f1 = wrapint(self.space, x)
> -        f2 = wrapint(self.space, y)
> -        v = self.space.or_(f1, f2)
> -        assert v.intval == x | y
> -
> -    def test_int(self):
> -        f1 = wrapint(self.space, 1)
> -        result = self.space.int(f1)
> -        assert result == f1
> -
> -    def test_oct(self):
> -        x = 012345
> -        f1 = wrapint(self.space, x)
> -        result = self.space.oct(f1)
> -        assert self.space.unwrap(result) == oct(x)
> -
> -    def test_hex(self):
> -        x = 0x12345
> -        f1 = wrapint(self.space, x)
> -        result = self.space.hex(f1)
> -        assert self.space.unwrap(result) == hex(x)
> -
> -
> -class AppTestSmallInt(AppTestInt):
> -    spaceconfig = {"objspace.std.optimized_int_add" : True,
> -                   "objspace.std.withsmallint" : True}
> diff --git a/pypy/objspace/std/typeobject.py b/pypy/objspace/std/typeobject.py
> --- a/pypy/objspace/std/typeobject.py
> +++ b/pypy/objspace/std/typeobject.py
> @@ -857,12 +857,6 @@
>   def _pure_issubtype(w_sub, w_type, version_tag1, version_tag2):
>       return _issubtype(w_sub, w_type)
>
> -def issubtype__Type_Type(space, w_type, w_sub):
> -    return space.newbool(w_sub.issubtype(w_type))
> -
> -def isinstance__Type_ANY(space, w_type, w_inst):
> -    return space.newbool(space.type(w_inst).issubtype(w_type))
> -
>   def repr__Type(space, w_obj):
>       w_mod = w_obj.get_module()
>       if not space.isinstance_w(w_mod, space.w_str):
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit
>


From fijall at gmail.com  Sat Mar 23 21:57:06 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 23 Mar 2013 13:57:06 -0700
Subject: [pypy-dev] [pypy-commit] pypy default: (fijal,
 alex) kill 'int' as a multimethod
In-Reply-To: <514E116B.9080205@gmx.de>
References: <20130323190209.6105D1C016E@cobra.cs.uni-duesseldorf.de>
	<514E116B.9080205@gmx.de>
Message-ID: <CAK5idxQKOFLsKjYebeNhgffXkORyw70cV7bxpkQtV9Gd8qwPew@mail.gmail.com>

On Sat, Mar 23, 2013 at 1:32 PM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> Hi Maciej, Alex,
>
> I really don't like this commit. It does two unrelated pretty
> fundamental changes together, killing small integers and making int a
> normal method. The first of these is not even explained in the commit
> message.
>
> I understand the reasons for removing small integers (they don't work
> with the JIT anyway, fixing that would be hard, ...). But it would
> still be good to have two commits, one that removes them, the other one
> that does your refactoring. This would give people that want to revive
> them at a later point a place to start.
>
> I don't actually know what should be done now. Backing out your change
> and doing it in two steps would be the right approach somehow, even if
> it requires some work.
>
> Cheers,
>
> Carl Friedrich
>

ok, fixed I think

>
>
> On 03/23/2013 08:02 PM, fijal wrote:
>>
>> Author: Maciej Fijalkowski <fijall at gmail.com>
>> Branch:
>> Changeset: r62700:e258f1470515
>> Date: 2013-03-23 12:01 -0700
>> http://bitbucket.org/pypy/pypy/changeset/e258f1470515/
>>
>> Log:    (fijal, alex) kill 'int' as a multimethod
>>
>> diff --git a/pypy/config/pypyoption.py b/pypy/config/pypyoption.py
>> --- a/pypy/config/pypyoption.py
>> +++ b/pypy/config/pypyoption.py
>> @@ -189,11 +189,6 @@
>>           BoolOption("withtproxy", "support transparent proxies",
>>                      default=True),
>>
>> -        BoolOption("withsmallint", "use tagged integers",
>> -                   default=False,
>> -                   requires=[("objspace.std.withprebuiltint", False),
>> -                             ("translation.taggedpointers", True)]),
>> -
>>           BoolOption("withprebuiltint", "prebuild commonly used int
>> objects",
>>                      default=False),
>>
>> @@ -204,9 +199,7 @@
>>                     default=100, cmdline="--prebuiltintto"),
>>
>>           BoolOption("withsmalllong", "use a version of 'long' in a C long
>> long",
>> -                   default=False,
>> -                   requires=[("objspace.std.withsmallint", False)]),
>> -                             #  ^^^ because of missing delegate_xx2yy
>> +                   default=False),
>>
>>           BoolOption("withstrbuf", "use strings optimized for addition
>> (ver 2)",
>>                      default=False),
>> diff --git a/pypy/interpreter/baseobjspace.py
>> b/pypy/interpreter/baseobjspace.py
>> --- a/pypy/interpreter/baseobjspace.py
>> +++ b/pypy/interpreter/baseobjspace.py
>> @@ -223,6 +223,22 @@
>>           raise OperationError(space.w_TypeError,
>>                                typed_unwrap_error_msg(space, "integer",
>> self))
>>
>> +    def int(self, space):
>> +        w_impl = space.lookup(self, '__int__')
>> +        if w_impl is None:
>> +            typename = space.type(self).getname(space)
>> +            raise operationerrfmt(space.w_TypeError,
>> +                  "unsupported operand type for int(): '%s'",
>> +                                  typename)
>> +        w_result = space.get_and_call_function(w_impl, self)
>> +
>> +        if (space.isinstance_w(w_result, space.w_int) or
>> +            space.isinstance_w(w_result, space.w_long)):
>> +            return w_result
>> +        typename = space.type(w_result).getname(space)
>> +        msg = "__int__ returned non-int (type '%s')"
>> +        raise operationerrfmt(space.w_TypeError, msg, typename)
>> +
>>       def __spacebind__(self, space):
>>           return self
>>
>> @@ -1306,6 +1322,9 @@
>>       def int_w(self, w_obj):
>>           return w_obj.int_w(self)
>>
>> +    def int(self, w_obj):
>> +        return w_obj.int(self)
>> +
>>       def uint_w(self, w_obj):
>>           return w_obj.uint_w(self)
>>
>> diff --git a/pypy/objspace/descroperation.py
>> b/pypy/objspace/descroperation.py
>> --- a/pypy/objspace/descroperation.py
>> +++ b/pypy/objspace/descroperation.py
>> @@ -808,7 +808,6 @@
>>   # more of the above manually-coded operations as well)
>>
>>   for targetname, specialname, checkerspec in [
>> -    ('int', '__int__', ("space.w_int", "space.w_long")),
>>       ('index', '__index__', ("space.w_int", "space.w_long")),
>>       ('long', '__long__', ("space.w_int", "space.w_long")),
>>       ('float', '__float__', ("space.w_float",))]:
>> diff --git a/pypy/objspace/std/complexobject.py
>> b/pypy/objspace/std/complexobject.py
>> --- a/pypy/objspace/std/complexobject.py
>> +++ b/pypy/objspace/std/complexobject.py
>> @@ -111,6 +111,9 @@
>>
>>           return w_result
>>
>> +    def int(self, space):
>> +        raise OperationError(space.w_TypeError, space.wrap("can't convert
>> complex to int; use int(abs(z))"))
>> +
>>   registerimplementation(W_ComplexObject)
>>
>>   w_one = W_ComplexObject(1, 0)
>> @@ -245,9 +248,6 @@
>>   def float__Complex(space, w_complex):
>>       raise OperationError(space.w_TypeError, space.wrap("can't convert
>> complex to float; use abs(z)"))
>>
>> -def int__Complex(space, w_complex):
>> -    raise OperationError(space.w_TypeError, space.wrap("can't convert
>> complex to int; use int(abs(z))"))
>> -
>>   def complex_conjugate__Complex(space, w_self):
>>       #w_real =
>> space.call_function(space.w_float,space.wrap(w_self.realval))
>>       #w_imag =
>> space.call_function(space.w_float,space.wrap(-w_self.imagval))
>> diff --git a/pypy/objspace/std/floatobject.py
>> b/pypy/objspace/std/floatobject.py
>> --- a/pypy/objspace/std/floatobject.py
>> +++ b/pypy/objspace/std/floatobject.py
>> @@ -58,6 +58,14 @@
>>       def float_w(self, space):
>>           return self.floatval
>>
>> +    def int(self, space):
>> +        try:
>> +            value = ovfcheck_float_to_int(self.floatval)
>> +        except OverflowError:
>> +            return space.long(self)
>> +        else:
>> +            return space.newint(value)
>> +
>>       def __repr__(self):
>>           return "<W_FloatObject(%f)>" % self.floatval
>>
>> @@ -89,14 +97,6 @@
>>       a = w_float1.floatval
>>       return W_FloatObject(a)
>>
>> -def int__Float(space, w_value):
>> -    try:
>> -        value = ovfcheck_float_to_int(w_value.floatval)
>> -    except OverflowError:
>> -        return space.long(w_value)
>> -    else:
>> -        return space.newint(value)
>> -
>>   def long__Float(space, w_floatobj):
>>       try:
>>           return W_LongObject.fromfloat(space, w_floatobj.floatval)
>> diff --git a/pypy/objspace/std/frame.py b/pypy/objspace/std/frame.py
>> --- a/pypy/objspace/std/frame.py
>> +++ b/pypy/objspace/std/frame.py
>> @@ -6,7 +6,7 @@
>>   from pypy.interpreter import pyopcode
>>   from pypy.interpreter.pyframe import PyFrame
>>   from pypy.interpreter.error import OperationError
>> -from pypy.objspace.std import intobject, smallintobject
>> +from pypy.objspace.std import intobject
>>   from pypy.objspace.std.multimethod import FailedToImplement
>>   from pypy.objspace.std.listobject import W_ListObject
>>
>> @@ -23,20 +23,6 @@
>>               raise AssertionError
>>
>>
>> -def small_int_BINARY_ADD(f, oparg, next_instr):
>> -    w_2 = f.popvalue()
>> -    w_1 = f.popvalue()
>> -    if (type(w_1) is smallintobject.W_SmallIntObject and
>> -        type(w_2) is smallintobject.W_SmallIntObject):
>> -        try:
>> -            w_result = smallintobject.add__SmallInt_SmallInt(f.space,
>> w_1, w_2)
>> -        except FailedToImplement:
>> -            w_result = f.space.add(w_1, w_2)
>> -    else:
>> -        w_result = f.space.add(w_1, w_2)
>> -    f.pushvalue(w_result)
>> -
>> -
>>   def int_BINARY_ADD(f, oparg, next_instr):
>>       w_2 = f.popvalue()
>>       w_1 = f.popvalue()
>> @@ -102,10 +88,7 @@
>>       class StdObjSpaceFrame(BaseFrame):
>>           pass
>>       if space.config.objspace.std.optimized_int_add:
>> -        if space.config.objspace.std.withsmallint:
>> -            StdObjSpaceFrame.BINARY_ADD = small_int_BINARY_ADD
>> -        else:
>> -            StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD
>> +        StdObjSpaceFrame.BINARY_ADD = int_BINARY_ADD
>>       if space.config.objspace.std.optimized_list_getitem:
>>           StdObjSpaceFrame.BINARY_SUBSCR = list_BINARY_SUBSCR
>>       if space.config.objspace.opcodes.CALL_METHOD:
>> diff --git a/pypy/objspace/std/intobject.py
>> b/pypy/objspace/std/intobject.py
>> --- a/pypy/objspace/std/intobject.py
>> +++ b/pypy/objspace/std/intobject.py
>> @@ -1,6 +1,6 @@
>>   from pypy.interpreter.error import OperationError
>>   from pypy.objspace.std import newformat
>> -from pypy.objspace.std.inttype import wrapint
>> +from pypy.objspace.std.inttype import wrapint, W_AbstractIntObject
>>   from pypy.objspace.std.model import registerimplementation, W_Object
>>   from pypy.objspace.std.multimethod import FailedToImplementArgs
>>   from pypy.objspace.std.noneobject import W_NoneObject
>> @@ -16,25 +16,6 @@
>>   something CPython does not do anymore.
>>   """
>>
>> -class W_AbstractIntObject(W_Object):
>> -    __slots__ = ()
>> -
>> -    def is_w(self, space, w_other):
>> -        if not isinstance(w_other, W_AbstractIntObject):
>> -            return False
>> -        if self.user_overridden_class or w_other.user_overridden_class:
>> -            return self is w_other
>> -        return space.int_w(self) == space.int_w(w_other)
>> -
>> -    def immutable_unique_id(self, space):
>> -        if self.user_overridden_class:
>> -            return None
>> -        from pypy.objspace.std.model import IDTAG_INT as tag
>> -        b = space.bigint_w(self)
>> -        b = b.lshift(3).or_(rbigint.fromint(tag))
>> -        return space.newlong_from_rbigint(b)
>> -
>> -
>>   class W_IntObject(W_AbstractIntObject):
>>       __slots__ = 'intval'
>>       _immutable_fields_ = ['intval']
>> @@ -67,6 +48,17 @@
>>       def float_w(self, space):
>>           return float(self.intval)
>>
>> +    def int(self, space):
>> +        # XXX find a better way to do it
>> +        if (type(self) != W_IntObject and
>> +            space.lookup(self, '__int__') is not
>> +            space.lookup_in_type_where(space.w_int, '__int__')[1]):
>> +            return W_Object.int(self, space)
>> +        if space.is_w(space.type(self), space.w_int):
>> +            return self
>> +        a = self.intval
>> +        return wrapint(space, a)
>> +
>>   registerimplementation(W_IntObject)
>>
>>   # NB: This code is shared by smallintobject.py, and thus no other Int
>> @@ -104,7 +96,7 @@
>>       # unlike CPython, we don't special-case the value -1 in most of our
>>       # hash functions, so there is not much sense special-casing it here
>> either.
>>       # Make sure this is consistent with the hash of floats and longs.
>> -    return get_integer(space, w_int1)
>> +    return w_int1.int(space)
>>
>>   # coerce
>>   def coerce__Int_Int(space, w_int1, w_int2):
>> @@ -251,7 +243,7 @@
>>
>>   def abs__Int(space, w_int1):
>>       if w_int1.intval >= 0:
>> -        return get_integer(space, w_int1)
>> +        return w_int1.int(space)
>>       else:
>>           return get_negint(space, w_int1)
>>
>> @@ -278,7 +270,7 @@
>>                                space.wrap("negative shift count"))
>>       else: #b >= LONG_BIT
>>           if a == 0:
>> -            return get_integer(space, w_int1)
>> +            return w_int1.int(space)
>>           raise FailedToImplementArgs(space.w_OverflowError,
>>                                   space.wrap("integer left shift"))
>>
>> @@ -291,7 +283,7 @@
>>                                    space.wrap("negative shift count"))
>>           else: # b >= LONG_BIT
>>               if a == 0:
>> -                return get_integer(space, w_int1)
>> +                return w_int1.int(space)
>>               if a < 0:
>>                   a = -1
>>               else:
>> @@ -321,17 +313,13 @@
>>   # int__Int is supposed to do nothing, unless it has
>>   # a derived integer object, where it should return
>>   # an exact one.
>> -def int__Int(space, w_int1):
>> -    if space.is_w(space.type(w_int1), space.w_int):
>> -        return w_int1
>> -    a = w_int1.intval
>> -    return wrapint(space, a)
>> -get_integer = int__Int
>> -pos__Int = int__Int
>> -trunc__Int = int__Int
>> +
>> +def pos__Int(self, space):
>> +    return self.int(space)
>> +trunc__Int = pos__Int
>>
>>   def index__Int(space, w_int1):
>> -    return get_integer(space, w_int1)
>> +    return w_int1.int(space)
>>
>>   def float__Int(space, w_int1):
>>       a = w_int1.intval
>> diff --git a/pypy/objspace/std/inttype.py b/pypy/objspace/std/inttype.py
>> --- a/pypy/objspace/std/inttype.py
>> +++ b/pypy/objspace/std/inttype.py
>> @@ -1,5 +1,6 @@
>>   from pypy.interpreter import typedef
>> -from pypy.interpreter.gateway import interp2app, unwrap_spec,
>> WrappedDefault
>> +from pypy.interpreter.gateway import interp2app, unwrap_spec,
>> WrappedDefault,\
>> +     interpindirect2app
>>   from pypy.interpreter.error import OperationError, operationerrfmt
>>   from pypy.interpreter.buffer import Buffer
>>   from pypy.objspace.std.register_all import register_all
>> @@ -7,8 +8,10 @@
>>   from pypy.objspace.std.strutil import (string_to_int, string_to_bigint,
>>                                          ParseStringError,
>>                                          ParseStringOverflowError)
>> +from pypy.objspace.std.model import W_Object
>>   from rpython.rlib.rarithmetic import r_uint
>>   from rpython.rlib.objectmodel import instantiate
>> +from rpython.rlib.rbigint import rbigint
>>
>>   # ____________________________________________________________
>>
>> @@ -36,14 +39,7 @@
>>
>>
>>   def wrapint(space, x):
>> -    if space.config.objspace.std.withsmallint:
>> -        from pypy.objspace.std.smallintobject import W_SmallIntObject
>> -        try:
>> -            return W_SmallIntObject(x)
>> -        except OverflowError:
>> -            from pypy.objspace.std.intobject import W_IntObject
>> -            return W_IntObject(x)
>> -    elif space.config.objspace.std.withprebuiltint:
>> +    if space.config.objspace.std.withprebuiltint:
>>           from pypy.objspace.std.intobject import W_IntObject
>>           lower = space.config.objspace.std.prebuiltintfrom
>>           upper =  space.config.objspace.std.prebuiltintto
>> @@ -185,6 +181,27 @@
>>
>>   # ____________________________________________________________
>>
>> +class W_AbstractIntObject(W_Object):
>> +    __slots__ = ()
>> +
>> +    def is_w(self, space, w_other):
>> +        if not isinstance(w_other, W_AbstractIntObject):
>> +            return False
>> +        if self.user_overridden_class or w_other.user_overridden_class:
>> +            return self is w_other
>> +        return space.int_w(self) == space.int_w(w_other)
>> +
>> +    def immutable_unique_id(self, space):
>> +        if self.user_overridden_class:
>> +            return None
>> +        from pypy.objspace.std.model import IDTAG_INT as tag
>> +        b = space.bigint_w(self)
>> +        b = b.lshift(3).or_(rbigint.fromint(tag))
>> +        return space.newlong_from_rbigint(b)
>> +
>> +    def int(self, space):
>> +        raise NotImplementedError
>> +
>>   int_typedef = StdTypeDef("int",
>>       __doc__ = '''int(x[, base]) -> integer
>>
>> @@ -201,5 +218,6 @@
>>       denominator = typedef.GetSetProperty(descr_get_denominator),
>>       real = typedef.GetSetProperty(descr_get_real),
>>       imag = typedef.GetSetProperty(descr_get_imag),
>> +    __int__ = interpindirect2app(W_AbstractIntObject.int),
>>   )
>>   int_typedef.registermethods(globals())
>> diff --git a/pypy/objspace/std/longobject.py
>> b/pypy/objspace/std/longobject.py
>> --- a/pypy/objspace/std/longobject.py
>> +++ b/pypy/objspace/std/longobject.py
>> @@ -87,6 +87,12 @@
>>       def float_w(self, space):
>>           return self.num.tofloat()
>>
>> +    def int(self, space):
>> +        try:
>> +            return space.newint(self.num.toint())
>> +        except OverflowError:
>> +            return long__Long(space, self)
>> +
>>       def __repr__(self):
>>           return '<W_LongObject(%d)>' % self.num.tolong()
>>
>> @@ -130,12 +136,6 @@
>>   def long__Int(space, w_intobj):
>>       return space.newlong(w_intobj.intval)
>>
>> -def int__Long(space, w_value):
>> -    try:
>> -        return space.newint(w_value.num.toint())
>> -    except OverflowError:
>> -        return long__Long(space, w_value)
>> -
>>   def index__Long(space, w_value):
>>       return long__Long(space, w_value)
>>
>> diff --git a/pypy/objspace/std/model.py b/pypy/objspace/std/model.py
>> --- a/pypy/objspace/std/model.py
>> +++ b/pypy/objspace/std/model.py
>> @@ -17,7 +17,6 @@
>>   option_to_typename = {
>>       "withspecialisedtuple" :
>> ["specialisedtupleobject.W_SpecialisedTupleObject"],
>>       "withsmalltuple" : ["smalltupleobject.W_SmallTupleObject"],
>> -    "withsmallint"   : ["smallintobject.W_SmallIntObject"],
>>       "withsmalllong"  : ["smalllongobject.W_SmallLongObject"],
>>       "withstrbuf"     : ["strbufobject.W_StringBufferObject"],
>>   }
>> @@ -158,18 +157,6 @@
>>           # when trying to dispatch multimethods.
>>           # XXX build these lists a bit more automatically later
>>
>> -        if config.objspace.std.withsmallint:
>> -            from pypy.objspace.std import smallintobject
>> -            self.typeorder[boolobject.W_BoolObject] += [
>> -                (smallintobject.W_SmallIntObject,
>> boolobject.delegate_Bool2SmallInt),
>> -                ]
>> -            self.typeorder[smallintobject.W_SmallIntObject] += [
>> -                (intobject.W_IntObject,
>> smallintobject.delegate_SmallInt2Int),
>> -                (floatobject.W_FloatObject,
>> smallintobject.delegate_SmallInt2Float),
>> -                (longobject.W_LongObject,
>> smallintobject.delegate_SmallInt2Long),
>> -                (complexobject.W_ComplexObject,
>> smallintobject.delegate_SmallInt2Complex),
>> -                ]
>> -
>>           if config.objspace.usemodules.micronumpy:
>>               from pypy.module.micronumpy.stdobjspace import
>> register_delegates
>>               register_delegates(self.typeorder)
>> @@ -424,7 +411,7 @@
>>       ['delattr', 'delete', 'get', 'id', 'inplace_div',
>> 'inplace_floordiv',
>>        'inplace_lshift', 'inplace_mod', 'inplace_pow', 'inplace_rshift',
>>        'inplace_truediv', 'is_', 'set', 'setattr', 'type', 'userdel',
>> -     'isinstance', 'issubtype'])
>> +     'isinstance', 'issubtype', 'int'])
>>   # XXX should we just remove those from the method table or we're happy
>>   #     with just not having multimethods?
>>
>> @@ -442,7 +429,7 @@
>>
>>       # add all regular multimethods here
>>       for _name, _symbol, _arity, _specialnames in ObjSpace.MethodTable:
>> -        if _name not in locals() or _name in NOT_MULTIMETHODS:
>> +        if _name not in locals() and _name not in NOT_MULTIMETHODS:
>>               mm = StdObjSpaceMultiMethod(_symbol, _arity, _specialnames)
>>               locals()[_name] = mm
>>               del mm
>> diff --git a/pypy/objspace/std/objspace.py b/pypy/objspace/std/objspace.py
>> --- a/pypy/objspace/std/objspace.py
>> +++ b/pypy/objspace/std/objspace.py
>> @@ -26,7 +26,6 @@
>>   from pypy.objspace.std.iterobject import W_SeqIterObject
>>   from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject
>>   from pypy.objspace.std.sliceobject import W_SliceObject
>> -from pypy.objspace.std.smallintobject import W_SmallIntObject
>>   from pypy.objspace.std.stringobject import W_StringObject
>>   from pypy.objspace.std.unicodeobject import W_UnicodeObject
>>   from pypy.objspace.std.tupleobject import W_AbstractTupleObject
>> @@ -580,16 +579,8 @@
>>               self.setitem(w_obj, self.wrap(key), w_value)
>>
>>       def getindex_w(self, w_obj, w_exception, objdescr=None):
>> -        # Performance shortcut for the common case of w_obj being an int.
>> -        # If withsmallint is disabled, we check for W_IntObject.
>> -        # If withsmallint is enabled, we only check for W_SmallIntObject
>> - it's
>> -        # probably not useful to have a shortcut for W_IntObject at all
>> then.
>> -        if self.config.objspace.std.withsmallint:
>> -            if type(w_obj) is W_SmallIntObject:
>> -                return w_obj.intval
>> -        else:
>> -            if type(w_obj) is W_IntObject:
>> -                return w_obj.intval
>> +        if type(w_obj) is W_IntObject:
>> +            return w_obj.intval
>>           return ObjSpace.getindex_w(self, w_obj, w_exception, objdescr)
>>
>>       def call_method(self, w_obj, methname, *arg_w):
>> diff --git a/pypy/objspace/std/smallintobject.py
>> b/pypy/objspace/std/smallintobject.py
>> deleted file mode 100644
>> --- a/pypy/objspace/std/smallintobject.py
>> +++ /dev/null
>> @@ -1,87 +0,0 @@
>> -"""
>> -Implementation of small ints, stored as odd-valued pointers in the
>> -translated PyPy.  To enable them, see inttype.py.
>> -"""
>> -from pypy.objspace.std import intobject
>> -from pypy.objspace.std.model import registerimplementation, W_Object
>> -from pypy.objspace.std.register_all import register_all
>> -from pypy.objspace.std.noneobject import W_NoneObject
>> -from pypy.objspace.std.intobject import W_AbstractIntObject, W_IntObject
>> -from pypy.interpreter.error import OperationError
>> -from rpython.rlib.objectmodel import UnboxedValue
>> -from rpython.rlib.rbigint import rbigint
>> -from rpython.rlib.rarithmetic import r_uint
>> -from rpython.tool.sourcetools import func_with_new_name
>> -from pypy.objspace.std.inttype import wrapint
>> -
>> -class W_SmallIntObject(W_AbstractIntObject, UnboxedValue):
>> -    __slots__ = 'intval'
>> -    from pypy.objspace.std.inttype import int_typedef as typedef
>> -
>> -    def unwrap(w_self, space):
>> -        return int(w_self.intval)
>> -    int_w = unwrap
>> -
>> -    def uint_w(w_self, space):
>> -        intval = w_self.intval
>> -        if intval < 0:
>> -            raise OperationError(space.w_ValueError,
>> -                                 space.wrap("cannot convert negative
>> integer to unsigned"))
>> -        else:
>> -            return r_uint(intval)
>> -
>> -    def bigint_w(w_self, space):
>> -        return rbigint.fromint(w_self.intval)
>> -
>> -
>> -registerimplementation(W_SmallIntObject)
>> -
>> -
>> -def delegate_SmallInt2Int(space, w_small):
>> -    return W_IntObject(w_small.intval)
>> -
>> -def delegate_SmallInt2Long(space, w_small):
>> -    return space.newlong(w_small.intval)
>> -
>> -def delegate_SmallInt2Float(space, w_small):
>> -    return space.newfloat(float(w_small.intval))
>> -
>> -def delegate_SmallInt2Complex(space, w_small):
>> -    return space.newcomplex(float(w_small.intval), 0.0)
>> -
>> -def add__SmallInt_SmallInt(space, w_a, w_b):
>> -    return wrapint(space, w_a.intval + w_b.intval) # cannot overflow
>> -
>> -def sub__SmallInt_SmallInt(space, w_a, w_b):
>> -    return wrapint(space, w_a.intval - w_b.intval) # cannot overflow
>> -
>> -def floordiv__SmallInt_SmallInt(space, w_a, w_b):
>> -    return wrapint(space, w_a.intval // w_b.intval) # cannot overflow
>> -
>> -div__SmallInt_SmallInt = floordiv__SmallInt_SmallInt
>> -
>> -def mod__SmallInt_SmallInt(space, w_a, w_b):
>> -    return wrapint(space, w_a.intval % w_b.intval) # cannot overflow
>> -
>> -def divmod__SmallInt_SmallInt(space, w_a, w_b):
>> -    w = wrapint(space, w_a.intval // w_b.intval) # cannot overflow
>> -    z = wrapint(space, w_a.intval % w_b.intval)
>> -    return space.newtuple([w, z])
>> -
>> -def copy_multimethods(ns):
>> -    """Copy integer multimethods for small int."""
>> -    for name, func in intobject.__dict__.iteritems():
>> -        if "__Int" in name:
>> -            new_name = name.replace("Int", "SmallInt")
>> -            if new_name not in ns:
>> -                # Copy the function, so the annotator specializes it for
>> -                # W_SmallIntObject.
>> -                ns[new_name] = func = func_with_new_name(func, new_name,
>> globals=ns)
>> -        else:
>> -            ns[name] = func
>> -    ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"]
>> -    ns["get_negint"] = ns["neg__SmallInt"]
>> -
>> -copy_multimethods(globals())
>> -
>> -register_all(vars())
>> diff --git a/pypy/objspace/std/smalllongobject.py
>> b/pypy/objspace/std/smalllongobject.py
>> --- a/pypy/objspace/std/smalllongobject.py
>> +++ b/pypy/objspace/std/smalllongobject.py
>> @@ -69,7 +69,15 @@
>>       def float_w(self, space):
>>           return float(self.longlong)
>>
>> +    def int(self, space):
>> +        a = self.longlong
>> +        b = intmask(a)
>> +        if b == a:
>> +            return space.newint(b)
>> +        else:
>> +            return self
>>
>> +
>>   registerimplementation(W_SmallLongObject)
>>
>>   # ____________________________________________________________
>> @@ -119,14 +127,6 @@
>>   def long__SmallLong(space, w_value):
>>       return w_value
>>
>> -def int__SmallLong(space, w_value):
>> -    a = w_value.longlong
>> -    b = intmask(a)
>> -    if b == a:
>> -        return space.newint(b)
>> -    else:
>> -        return w_value
>> -
>>   def index__SmallLong(space, w_value):
>>       return w_value
>>
>> diff --git a/pypy/objspace/std/test/test_intobject.py
>> b/pypy/objspace/std/test/test_intobject.py
>> --- a/pypy/objspace/std/test/test_intobject.py
>> +++ b/pypy/objspace/std/test/test_intobject.py
>> @@ -268,7 +268,7 @@
>>
>>       def test_int(self):
>>           f1 = iobj.W_IntObject(1)
>> -        result = iobj.int__Int(self.space, f1)
>> +        result = f1.int(self.space)
>>           assert result == f1
>>
>>       def test_oct(self):
>> diff --git a/pypy/objspace/std/test/test_smallintobject.py
>> b/pypy/objspace/std/test/test_smallintobject.py
>> deleted file mode 100644
>> --- a/pypy/objspace/std/test/test_smallintobject.py
>> +++ /dev/null
>> @@ -1,229 +0,0 @@
>> -import sys, py
>> -
>> -#from pypy.objspace.std.model import WITHSMALLINT
>> -#if not WITHSMALLINT:
>> -#    py.test.skip("WITHSMALLINT is not enabled")
>> -
>> -from pypy.objspace.std.inttype import wrapint
>> -from pypy.objspace.std.multimethod import FailedToImplement
>> -from rpython.rlib.rarithmetic import r_uint
>> -
>> -from pypy.objspace.std.test.test_intobject import AppTestInt
>> -
>> -class TestW_IntObject:
>> -    spaceconfig = {"objspace.std.withsmallint": True}
>> -
>> -    def test_int_w(self):
>> -        assert self.space.int_w(self.space.wrap(42)) == 42
>> -
>> -    def test_uint_w(self):
>> -        space = self.space
>> -        assert space.uint_w(space.wrap(42)) == 42
>> -        assert isinstance(space.uint_w(space.wrap(42)), r_uint)
>> -        space.raises_w(space.w_ValueError, space.uint_w, space.wrap(-1))
>> -
>> -    def test_repr(self):
>> -        x = 1
>> -        f1 = wrapint(self.space, x)
>> -        result = self.space.repr(f1)
>> -        assert self.space.unwrap(result) == repr(x)
>> -
>> -    def test_str(self):
>> -        x = 12345
>> -        f1 = wrapint(self.space, x)
>> -        result = self.space.str(f1)
>> -        assert self.space.unwrap(result) == str(x)
>> -
>> -    def test_hash(self):
>> -        x = 42
>> -        f1 = wrapint(self.space, x)
>> -        result = self.space.hash(f1)
>> -        assert result.intval == hash(x)
>> -
>> -    def test_compare(self):
>> -        import operator
>> -        optab = ['lt', 'le', 'eq', 'ne', 'gt', 'ge']
>> -        for x in (-10, -1, 0, 1, 2, 1000, sys.maxint):
>> -            for y in (-sys.maxint-1, -11, -9, -2, 0, 1, 3, 1111,
>> sys.maxint):
>> -                for op in optab:
>> -                    wx = wrapint(self.space, x)
>> -                    wy = wrapint(self.space, y)
>> -                    res = getattr(operator, op)(x, y)
>> -                    method = getattr(self.space, op)
>> -                    myres = method(wx, wy)
>> -                    assert self.space.unwrap(myres) == res
>> -
>> -    def test_add(self):
>> -        for x in [1, 100, sys.maxint // 2 - 50,
>> -                  sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
>> -            for y in [1, 100, sys.maxint // 2 - 50,
>> -                      sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
>> -                f1 = wrapint(self.space, x)
>> -                f2 = wrapint(self.space, y)
>> -                result = self.space.unwrap(self.space.add(f1, f2))
>> -                assert result == x+y
>> -
>> -    def test_sub(self):
>> -        for x in [1, 100, sys.maxint // 2 - 50,
>> -                  sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
>> -            for y in [1, 100, sys.maxint // 2 - 50,
>> -                      sys.maxint // 2, sys.maxint - 1000, sys.maxint]:
>> -                f1 = wrapint(self.space, x)
>> -                f2 = wrapint(self.space, y)
>> -                result = self.space.unwrap(self.space.sub(f1, f2))
>> -                assert result == x-y
>> -
>> -    def test_mul(self):
>> -        for x in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint - 1000]:
>> -            for y in [0, 1, 100, sys.maxint // 2 - 50, sys.maxint -
>> 1000]:
>> -                f1 = wrapint(self.space, x)
>> -                f2 = wrapint(self.space, y)
>> -                result = self.space.unwrap(self.space.mul(f1, f2))
>> -                assert result == x*y
>> -
>> -
>> -    def test_div(self):
>> -        for i in range(10):
>> -            res = i//3
>> -            f1 = wrapint(self.space, i)
>> -            f2 = wrapint(self.space, 3)
>> -            result = self.space.div(f1, f2)
>> -            assert result.intval == res
>> -
>> -    def test_mod(self):
>> -        x = 1
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.mod(f1, f2)
>> -        assert v.intval == x % y
>> -        # not that mod cannot overflow
>> -
>> -    def test_divmod(self):
>> -        x = 1
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        ret = self.space.divmod(f1, f2)
>> -        v, w = self.space.unwrap(ret)
>> -        assert (v, w) == divmod(x, y)
>> -
>> -    def test_pow_iii(self):
>> -        x = 10
>> -        y = 2
>> -        z = 13
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        f3 = wrapint(self.space, z)
>> -        v = self.space.pow(f1, f2, f3)
>> -        assert v.intval == pow(x, y, z)
>> -        f1, f2, f3 = [wrapint(self.space, i) for i in (10, -1, 42)]
>> -        self.space.raises_w(self.space.w_TypeError,
>> -                            self.space.pow,
>> -                            f1, f2, f3)
>> -        f1, f2, f3 = [wrapint(self.space, i) for i in (10, 5, 0)]
>> -        self.space.raises_w(self.space.w_ValueError,
>> -                            self.space.pow,
>> -                            f1, f2, f3)
>> -
>> -    def test_pow_iin(self):
>> -        x = 10
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.pow(f1, f2, self.space.w_None)
>> -        assert v.intval == x ** y
>> -
>> -    def test_neg(self):
>> -        x = 42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.neg(f1)
>> -        assert v.intval == -x
>> -
>> -    def test_pos(self):
>> -        x = 42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.pos(f1)
>> -        assert v.intval == +x
>> -        x = -42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.pos(f1)
>> -        assert v.intval == +x
>> -
>> -    def test_abs(self):
>> -        x = 42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.abs(f1)
>> -        assert v.intval == abs(x)
>> -        x = -42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.abs(f1)
>> -        assert v.intval == abs(x)
>> -
>> -    def test_invert(self):
>> -        x = 42
>> -        f1 = wrapint(self.space, x)
>> -        v = self.space.invert(f1)
>> -        assert v.intval == ~x
>> -
>> -    def test_lshift(self):
>> -        x = 12345678
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.lshift(f1, f2)
>> -        assert v.intval == x << y
>> -
>> -    def test_rshift(self):
>> -        x = 12345678
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.rshift(f1, f2)
>> -        assert v.intval == x >> y
>> -
>> -    def test_and(self):
>> -        x = 12345678
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.and_(f1, f2)
>> -        assert v.intval == x & y
>> -
>> -    def test_xor(self):
>> -        x = 12345678
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.xor(f1, f2)
>> -        assert v.intval == x ^ y
>> -
>> -    def test_or(self):
>> -        x = 12345678
>> -        y = 2
>> -        f1 = wrapint(self.space, x)
>> -        f2 = wrapint(self.space, y)
>> -        v = self.space.or_(f1, f2)
>> -        assert v.intval == x | y
>> -
>> -    def test_int(self):
>> -        f1 = wrapint(self.space, 1)
>> -        result = self.space.int(f1)
>> -        assert result == f1
>> -
>> -    def test_oct(self):
>> -        x = 012345
>> -        f1 = wrapint(self.space, x)
>> -        result = self.space.oct(f1)
>> -        assert self.space.unwrap(result) == oct(x)
>> -
>> -    def test_hex(self):
>> -        x = 0x12345
>> -        f1 = wrapint(self.space, x)
>> -        result = self.space.hex(f1)
>> -        assert self.space.unwrap(result) == hex(x)
>> -
>> -
>> -class AppTestSmallInt(AppTestInt):
>> -    spaceconfig = {"objspace.std.optimized_int_add" : True,
>> -                   "objspace.std.withsmallint" : True}
>> diff --git a/pypy/objspace/std/typeobject.py
>> b/pypy/objspace/std/typeobject.py
>> --- a/pypy/objspace/std/typeobject.py
>> +++ b/pypy/objspace/std/typeobject.py
>> @@ -857,12 +857,6 @@
>>   def _pure_issubtype(w_sub, w_type, version_tag1, version_tag2):
>>       return _issubtype(w_sub, w_type)
>>
>> -def issubtype__Type_Type(space, w_type, w_sub):
>> -    return space.newbool(w_sub.issubtype(w_type))
>> -
>> -def isinstance__Type_ANY(space, w_type, w_inst):
>> -    return space.newbool(space.type(w_inst).issubtype(w_type))
>> -
>>   def repr__Type(space, w_obj):
>>       w_mod = w_obj.get_module()
>>       if not space.isinstance_w(w_mod, space.w_str):
>> _______________________________________________
>> pypy-commit mailing list
>> pypy-commit at python.org
>> http://mail.python.org/mailman/listinfo/pypy-commit
>>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From fijall at gmail.com  Sat Mar 23 22:02:52 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 23 Mar 2013 14:02:52 -0700
Subject: [pypy-dev] Experimental GIL
In-Reply-To: <CAE3+++QL3XTULFFqNfq5ORdi7Ffsw84b_xs+usoSgYG_Q9rQ4Q@mail.gmail.com>
References: <CAE3+++QL3XTULFFqNfq5ORdi7Ffsw84b_xs+usoSgYG_Q9rQ4Q@mail.gmail.com>
Message-ID: <CAK5idxQKsD9kAt2M3C_aMdMcCnRpOhb2q0MCN0qBYc8yUJeTRA@mail.gmail.com>

On Sat, Mar 23, 2013 at 1:07 PM, Marko Tasic <mtasic85 at gmail.com> wrote:
> Hi,
>
> I have had an idea for some time to re-implement how GIL works, but
> nothing radical. On purpose I'm saying that I don't want to "remove"
> GIL, but to modify it, so people who follow your list don't get too
> excited.
>
> Let me describe what I want to accomplish, and what kind of results am
> looking for, and also hear any possible design or implementation issue
> I/we may face.
>
> I want threads to work in parallel as they already do, with very
> little waiting times on other threads in case of GIL'ing. Actually, I
> want to use time reserved for GIL to do something else.
>
> To be clear, I don't need JIT, and lets take it out of discussion for
> some time later.
>
> First step:
> -------------
> My solution require wait-free lock-free queue, so many threads can
> only append/push to it atomically. Every thread created, including
> main thread, has to have (at least one) such a queue, and lets say
> queue has pointer to thread where it belongs.
>
> Not only queue is required, but also conditional variable (such as
> pthread_cond_t) per thread.
>
> What I imagine is high-level thread object that consists of low-level
> thread, low-level wait-free lock-free queue, and low-level conditional
> variable. Imagine that this high-level thread object is local per each
> OS thread created. From now on I will call high-level thread object
> just "thread". Every mutable object created has to have pointer to a
> thread in which its created. As a result, this way we can know origin
> of a mutable object by thread by inspecting mutable object itself, so
> as a consequence also know low-level thread, low-level queue and
> low-level condition variable associated by it.

You should look more at rust model where "mutable objects" need token
to read/write. In python it's an utter mess, because all classes are
mutable, so you need locking for object instantiation (or some sort of
token-passing) or something. This is a major nightmare.

>
> Having this special field is naive approach, so of course we can have
> allocation areas per thread, and GC can be smarter, but lets just
> forget about it for now.
>
> Second step:
> -----------------
> Every time "ceval" equivalent function observes a mutable object on
> bytecode interpretation, it checks if local thread is same as object's
> thread.
>
> If they are same, then proceed with interpretation of local ceval operation.
>
> If they are different, then:
>     - lets call local thread just T
>     - lets call operation just O
>     - take low-level queue of object's thread, lets call it Q
>     - group O and T into item and push it to Q
>     - set T's condition variable to wait
>
> Third step:
> -------------
> Lets imagine that we have old style GIL that counts opcodes executed,
> and after N opcodes instead of acquiring GIL, it processes queue. This
> processing is done by popping out operation (O) and thread (T) items
> from local thread's queue, and serially executing operations. After
> each executed operation it sends signal to condition variable passed
> inside thread which was passed in same queue item where is operation.
> Once, signal is received by awaiting thread, until now awaiting thread
> proceeds with execution, while thread that was processing queue also
> continues executing queued operations.
>
> Conclusion:
> ---------------
> There is ceval per thread. All ceval's run at same time. In the best
> case, they don't wait on each other because threads do not touch
> objects created in other threads. In the worst case, there is always
> only one thread executing operation, and all other are waiting it to
> finish, and this is close to how GIL already works (almost).
>
> Special care has to be taken for threads that are going to finish, and
> what happens with their objects if they are referenced by other
> objects that belong to other threads.

For what is worth this is *hard* - you need to write a new GC for starter.

>
> So I have also some questions:
> - How flexible is to modify GIL in pypy?

very, it's a translation-transformation.

> - Where should I look?

someone who calls objectmodel.invoke_around_extcall, from module/thread

> - Do I need to fully translate pypy in order to test it?

no, you can experiment on smaller languages (we do it all the time for
JIT, GC etc.)

> - I need wait-free look-free queue, so what kind of low-level atomic
> primitives do RPython has to offer?

none of what you want. you need to implement it.

> - I need to have special field for all mutable objects, so how to do so?

you can do this by judging in rtyper "are all the fields immutable".

>
> I do not expect any performance boost, just correctness of approach.
> Perhaps, I'm wrong.
>
> What do you think, is this practical approach?

it requires lots of work, but can in theory work. The token-passing
for mutable objects puts a heavy burden on say allocations or global
lookups. I don't think you would end up with a better performance than
STM, although feel free to try.

Cheers,
fijal

From alex.gaynor at gmail.com  Sat Mar 23 22:58:57 2013
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Sat, 23 Mar 2013 14:58:57 -0700
Subject: [pypy-dev] GSOC 2013
Message-ID: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>

Hi everyone,

I've started putting together an ideas page for this GSOC:
https://bitbucket.org/pypy/pypy/wiki/GSOC%202013

If you've got an idea, or are interested in mentoring you should add
yourself/the idea.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130323/bb2fbe37/attachment.html>

From sdouche at gmail.com  Mon Mar 25 07:28:48 2013
From: sdouche at gmail.com (Sebastien Douche)
Date: Mon, 25 Mar 2013 07:28:48 +0100
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
Message-ID: <CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>

On Sat, Mar 23, 2013 at 10:58 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
> If you've got an idea, or are interested in mentoring you should add
> yourself/the idea.

How is the state of the STM branch? Just a POC or Armin has a idea for
producing a new version of PyPy?

--
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From cfbolz at gmx.de  Mon Mar 25 13:26:22 2013
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 25 Mar 2013 13:26:22 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: use
 jit.loop_unrolling_heuristic where possible
In-Reply-To: <20130325094239.C28781C00E4@cobra.cs.uni-duesseldorf.de>
References: <20130325094239.C28781C00E4@cobra.cs.uni-duesseldorf.de>
Message-ID: <5150426E.3090906@gmx.de>

Hi Brian,

On 03/25/2013 10:42 AM, bdkearns wrote:
> Author: Brian Kearns <bdkearns at gmail.com>
> Branch:
> Changeset: r62735:1f7ca87c6780
> Date: 2013-03-25 05:37 -0400
> http://bitbucket.org/pypy/pypy/changeset/1f7ca87c6780/
>
> Log:	use jit.loop_unrolling_heuristic where possible
>
> diff --git a/pypy/module/__builtin__/functional.py b/pypy/module/__builtin__/functional.py
> --- a/pypy/module/__builtin__/functional.py
> +++ b/pypy/module/__builtin__/functional.py
> @@ -202,8 +202,8 @@
>
>   @specialize.arg(2)
>   def min_max(space, args, implementation_of):
> -    if not jit.we_are_jitted() or (jit.isconstant(len(args.arguments_w)) and
> -            len(args.arguments_w) == 2):
> +    if not jit.we_are_jitted() or jit.loop_unrolling_heuristic(
> +            args.arguments_w, len(args.arguments_w)):
>           return min_max_unroll(space, args, implementation_of)
>       else:
>           return min_max_normal(space, args, implementation_of)

This is a very risky change. The unrolled pseudo-code for min (max is 
equivalent) with 5 elements is something like this:

m = l0
if l1 < m:
     m = l1
if l2 < m
     m = l2
if l3 < m
     m = l3
if l3 < m
     m = l3

(the comparisons of course have to go through the object space,
everything is more complex, but the control flow is essentially like
this.)

Every comparison can go either way, that means that there are 2**4
paths through this code, and it's completely unpredictable which of the
paths is taken. Therefore it can lead to potentially too much code
being generated. I think for min and max in particular the compromise
of just unrolling twice is acceptable.

BTW, another optimization that might be interesting for min/max is to
add fast paths with listview_int/listview_str if no key and no
comparison are given.


Cheers,

Carl Friedrich

> diff --git a/pypy/objspace/std/dictmultiobject.py b/pypy/objspace/std/dictmultiobject.py
> --- a/pypy/objspace/std/dictmultiobject.py
> +++ b/pypy/objspace/std/dictmultiobject.py
> @@ -10,8 +10,7 @@
>   from rpython.rlib.debug import mark_dict_non_null
>   from rpython.tool.sourcetools import func_with_new_name
>
> -from rpython.rlib import rerased
> -from rpython.rlib import jit
> +from rpython.rlib import rerased, jit
>
>   def _is_str(space, w_key):
>       return space.is_w(space.type(w_key), space.w_str)
> @@ -29,9 +28,6 @@
>               space.is_w(w_lookup_type, space.w_float)
>               )
>
> -
> -DICT_CUTOFF = 5
> -
>   @specialize.call_location()
>   def w_dict_unrolling_heuristic(w_dct):
>       """ In which cases iterating over dict items can be unrolled.
> @@ -39,7 +35,8 @@
>       an actual dict
>       """
>       return jit.isvirtual(w_dct) or (jit.isconstant(w_dct) and
> -                                    w_dct.length() <= DICT_CUTOFF)
> +                                    w_dct.length() <= jit.UNROLL_CUTOFF)
> +
>
>   class W_DictMultiObject(W_Object):
>       from pypy.objspace.std.dicttype import dict_typedef as typedef
> @@ -761,7 +758,8 @@
>               update1_keys(space, w_dict, w_data)
>
>
> - at jit.look_inside_iff(lambda space, w_dict, w_data: w_dict_unrolling_heuristic(w_data))
> + at jit.look_inside_iff(lambda space, w_dict, w_data:
> +                     w_dict_unrolling_heuristic(w_data))
>   def update1_dict_dict(space, w_dict, w_data):
>       iterator = w_data.iteritems()
>       while 1:
> diff --git a/pypy/objspace/std/intobject.py b/pypy/objspace/std/intobject.py
> --- a/pypy/objspace/std/intobject.py
> +++ b/pypy/objspace/std/intobject.py
> @@ -183,7 +183,8 @@
>
>
>   # helper for pow()
> - at jit.look_inside_iff(lambda space, iv, iw, iz: jit.isconstant(iw) and jit.isconstant(iz))
> + at jit.look_inside_iff(lambda space, iv, iw, iz:
> +                     jit.isconstant(iw) and jit.isconstant(iz))
>   def _impl_int_int_pow(space, iv, iw, iz):
>       if iw < 0:
>           if iz != 0:
> diff --git a/pypy/objspace/std/kwargsdict.py b/pypy/objspace/std/kwargsdict.py
> --- a/pypy/objspace/std/kwargsdict.py
> +++ b/pypy/objspace/std/kwargsdict.py
> @@ -95,7 +95,8 @@
>       def getitem_str(self, w_dict, key):
>           return self._getitem_str_indirection(w_dict, key)
>
> -    @jit.look_inside_iff(lambda self, w_dict, key: jit.isconstant(self.length(w_dict)) and jit.isconstant(key))
> +    @jit.look_inside_iff(lambda self, w_dict, key:
> +            jit.isconstant(self.length(w_dict)) and jit.isconstant(key))
>       def _getitem_str_indirection(self, w_dict, key):
>           keys, values_w = self.unerase(w_dict.dstorage)
>           result = []
> diff --git a/pypy/objspace/std/listobject.py b/pypy/objspace/std/listobject.py
> --- a/pypy/objspace/std/listobject.py
> +++ b/pypy/objspace/std/listobject.py
> @@ -18,7 +18,6 @@
>   from pypy.objspace.std.stdtypedef import StdTypeDef, SMM
>   from sys import maxint
>
> -UNROLL_CUTOFF = 5
>
>   class W_AbstractListObject(W_Object):
>       __slots__ = ()
> @@ -43,7 +42,7 @@
>       return W_ListObject.from_storage_and_strategy(space, storage, strategy)
>
>   @jit.look_inside_iff(lambda space, list_w, sizehint:
> -                         jit.isconstant(len(list_w)) and len(list_w) < UNROLL_CUTOFF)
> +                     jit.loop_unrolling_heuristic(list_w, len(list_w)))
>   def get_strategy_from_list_objects(space, list_w, sizehint):
>       if not list_w:
>           if sizehint != -1:
> @@ -950,7 +949,7 @@
>           raise NotImplementedError("abstract base class")
>
>       @jit.look_inside_iff(lambda space, w_list, list_w:
> -        jit.isconstant(len(list_w)) and len(list_w) < UNROLL_CUTOFF)
> +                         jit.loop_unrolling_heuristic(list_w, len(list_w)))
>       def init_from_list_w(self, w_list, list_w):
>           l = [self.unwrap(w_item) for w_item in list_w]
>           w_list.lstorage = self.erase(l)
> @@ -999,7 +998,7 @@
>           return self.wrap(r)
>
>       @jit.look_inside_iff(lambda self, w_list:
> -           jit.isconstant(w_list.length()) and w_list.length() < UNROLL_CUTOFF)
> +                         jit.loop_unrolling_heuristic(w_list, w_list.length()))
>       def getitems_copy(self, w_list):
>           return [self.wrap(item) for item in self.unerase(w_list.lstorage)]
>
> @@ -1008,7 +1007,7 @@
>           return [self.wrap(item) for item in self.unerase(w_list.lstorage)]
>
>       @jit.look_inside_iff(lambda self, w_list:
> -           jit.isconstant(w_list.length()) and w_list.length() < UNROLL_CUTOFF)
> +                         jit.loop_unrolling_heuristic(w_list, w_list.length()))
>       def getitems_fixedsize(self, w_list):
>           return self.getitems_unroll(w_list)
>
> diff --git a/pypy/objspace/std/tupleobject.py b/pypy/objspace/std/tupleobject.py
> --- a/pypy/objspace/std/tupleobject.py
> +++ b/pypy/objspace/std/tupleobject.py
> @@ -10,8 +10,6 @@
>   from rpython.rlib import jit
>   from rpython.tool.sourcetools import func_with_new_name
>
> -# Tuples of known length up to UNROLL_TUPLE_LIMIT have unrolled certain methods
> -UNROLL_TUPLE_LIMIT = 10
>
>   class W_AbstractTupleObject(W_Object):
>       __slots__ = ()
> @@ -85,15 +83,8 @@
>       start, stop = normalize_simple_slice(space, length, w_start, w_stop)
>       return space.newtuple(w_tuple.wrappeditems[start:stop])
>
> -THRESHOLD = 7
> -
> -def unroll_tuple_contains(space, w_tuple, w_obj):
> -    if (jit.isconstant(w_tuple) or jit.isvirtual(w_tuple) and
> -        len(w_tuple.wrappeditems) < THRESHOLD):
> -        return True
> -    return False
> -
> - at jit.look_inside_iff(unroll_tuple_contains)
> + at jit.look_inside_iff(lambda space, w_tuple, w_obj:
> +        jit.loop_unrolling_heuristic(w_tuple, len(w_tuple.wrappeditems)))
>   def contains__Tuple_ANY(space, w_tuple, w_obj):
>       for w_item in w_tuple.wrappeditems:
>           if space.eq_w(w_item, w_obj):
> @@ -128,10 +119,8 @@
>       return mul_tuple_times(space, w_tuple, w_times)
>
>   def tuple_unroll_condition(space, w_tuple1, w_tuple2):
> -    lgt1 = len(w_tuple1.wrappeditems)
> -    lgt2 = len(w_tuple2.wrappeditems)
> -    return ((jit.isconstant(lgt1) and lgt1 <= UNROLL_TUPLE_LIMIT) or
> -            (jit.isconstant(lgt2) and lgt2 <= UNROLL_TUPLE_LIMIT))
> +    return jit.loop_unrolling_heuristic(w_tuple1, len(w_tuple1.wrappeditems)) or \
> +           jit.loop_unrolling_heuristic(w_tuple2, len(w_tuple2.wrappeditems))
>
>   @jit.look_inside_iff(tuple_unroll_condition)
>   def eq__Tuple_Tuple(space, w_tuple1, w_tuple2):
> @@ -151,7 +140,7 @@
>   def _make_tuple_comparison(name):
>       import operator
>       op = getattr(operator, name)
> -    #
> +
>       @jit.look_inside_iff(tuple_unroll_condition)
>       def compare_tuples(space, w_tuple1, w_tuple2):
>           items1 = w_tuple1.wrappeditems
> @@ -184,8 +173,7 @@
>       return space.wrap(hash_tuple(space, w_tuple.wrappeditems))
>
>   @jit.look_inside_iff(lambda space, wrappeditems:
> -                     jit.isconstant(len(wrappeditems)) and
> -                     len(wrappeditems) < UNROLL_TUPLE_LIMIT)
> +        jit.loop_unrolling_heuristic(wrappeditems, len(wrappeditems)))
>   def hash_tuple(space, wrappeditems):
>       # this is the CPython 2.4 algorithm (changed from 2.3)
>       mult = 1000003
> diff --git a/rpython/rlib/jit.py b/rpython/rlib/jit.py
> --- a/rpython/rlib/jit.py
> +++ b/rpython/rlib/jit.py
> @@ -206,13 +206,13 @@
>       return NonConstant(False)
>   isvirtual._annspecialcase_ = "specialize:call_location"
>
> -LIST_CUTOFF = 2
> +UNROLL_CUTOFF = 5
>
>   @specialize.call_location()
>   def loop_unrolling_heuristic(lst, size):
>       """ In which cases iterating over items of lst can be unrolled
>       """
> -    return isvirtual(lst) or (isconstant(size) and size <= LIST_CUTOFF)
> +    return isvirtual(lst) or (isconstant(size) and size <= UNROLL_CUTOFF)
>
>   class Entry(ExtRegistryEntry):
>       _about_ = hint
> _______________________________________________
> pypy-commit mailing list
> pypy-commit at python.org
> http://mail.python.org/mailman/listinfo/pypy-commit
>
	

From techtonik at gmail.com  Mon Mar 25 18:05:49 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Mon, 25 Mar 2013 20:05:49 +0300
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAGmFidaQ2ja0rd=Ze7Lx03uQ7UofPXWDt5j0_4QqEpC81xSheQ@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
	<CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
	<CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>
	<CAPkN8xJvc08rVnEzuj7XCbR20uvk4wx=M1Qy+UuwidpC-ZZWPg@mail.gmail.com>
	<CAGmFidaQ2ja0rd=Ze7Lx03uQ7UofPXWDt5j0_4QqEpC81xSheQ@mail.gmail.com>
Message-ID: <CAPkN8xLMd-KjbXBuOrnjRipbKnfbVG9WSvzQNN33AiQoyH6ykg@mail.gmail.com>

On Wed, Mar 20, 2013 at 8:21 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>
>> On Wed, Mar 20, 2013 at 7:36 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com
>> > wrote:
>>
>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>
>>>> On Wed, Mar 20, 2013 at 5:28 PM, Amaury Forgeot d'Arc <
>>>> amauryfa at gmail.com> wrote:
>>>>
>>>>> 2013/3/20 anatoly techtonik <techtonik at gmail.com>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I've created a module to dump function trace during execution of
>>>>>> Python script. You can see session example in attachment. The format is of
>>>>>> PHP Xdebug tool [2] just because I had some scripts from the past to
>>>>>> analyze it.
>>>>>>
>>>>>> The module uses sys.settrace() to analyse frames in 'call' events
>>>>>> with callback(frame, event, arg).
>>>>>>
>>>>>> Recently I've got an anonymous report that some frame misses filename
>>>>>> information when run under IDE:
>>>>>>
>>>>>> https://bitbucket.org/techtonik/xtrace/issue/2/use-of-xtrace-module-within-ide-causes
>>>>>>
>>>>>> There is unlikely to be any more feedback from the user to make me
>>>>>> understand and reproduce the behavior, nor there is a sufficient
>>>>>> documentation in sys.trace description [1]. So I am on my own. But I can
>>>>>> not read C code the CPython is written in, so I ask here.
>>>>>>
>>>>>> What are possible execution contexts for Python?
>>>>>> How each execution context affects data structure that is passed to
>>>>>> trace function (frame and arg)?
>>>>>>
>>>>>
>>>>>  It's simpler than that: when running from an interactive session,
>>>>> f_back is empty...
>>>>>
>>>>> >>> def trace(frame, event, arg):
>>>>> ...     funcname = frame.f_code.co_name
>>>>> ...     if funcname == '<module>':
>>>>> ...         print frame.f_back.f_code.co_filename
>>>>> ...
>>>>> >>> import sys
>>>>> >>> sys.settrace(trace)
>>>>> >>> 1
>>>>> Traceback (most recent call last):
>>>>>   File "<stdin>", line 1, in <module>
>>>>>   File "<stdin>", line 4, in trace
>>>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>>>
>>>>
>>>> Thanks a lot. =)
>>>>
>>>> User said he was using module from the IDE and at first I thought it
>>>> wasn't possible to use module without previous stack frame (f_back), but I
>>>> was wrong:
>>>>
>>>> > py -2 -s -S
>>>> Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
>>>> (Intel)] on win32
>>>> >>> import xtrace
>>>> >>> xtrace.start()
>>>> TRACE START [2013-03-20 18:49:27]
>>>> >>> z = 0
>>>>   -> decode() C:\Python27\lib\encodings\cp437.py:14
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>>   File "xtrace.py", line 66, in function_trace_xdebug
>>>>     filename = self._strip_cwd(frame.f_back.f_code.co_filename)
>>>> AttributeError: 'NoneType' object has no attribute 'f_code'
>>>>
>>>>
>>>> On the way I've discovered that my Python is hacked. Without -s -S key
>>>> it executes modules from local temp directory.
>>>> ...
>>>>           -> ensure_unicode()
>>>> c:\users\user\appdata\local\temp\easy_install-k8gvbp\pyreadline-1.7.1-py2.7-win32.egg.tmp\pyre
>>>> dline\unicode_helper.py:20
>>>> ...
>>>>
>>>>
>>>> So there are absolutely no differences in running code in these
>>>> execution contexts?
>>>> - code runs from a file
>>>> - code runs from a interactive console
>>>>
>>>
>>> Of course there is a difference; the code below works run from a file,
>>> but not when pasted into the interactive console:
>>>
>>> def trace(frame, event, arg):
>>>     funcname = frame.f_code.co_name
>>>     print frame.f_code, repr(frame.f_code.co_code), event, arg
>>>     if funcname == '<module>':
>>>         print "    module", repr(frame.f_back.f_code.co_code)
>>>
>>> import sys
>>> sys.settrace(trace)
>>> exec '1 + 1'
>>>
>>
>>  For console it pretty clear why no previous frame exists - stack is
>> resolved at >>> prompt. But what is this top frame that appears inside file
>> context? The following code gives:
>> <module> 7
>> <module> 1
>>
>> def trace(frame, event, arg):
>>     print frame.f_back.f_code.co_name, frame.f_back.f_lineno
>>     print frame.f_code.co_name, frame.f_lineno
>>
>> import sys
>> sys.settrace(trace)
>> exec '1 + 1'
>>
>> How to identify the frames here? How come that 7th line of module is
>> executed before 1 st? I expected to have module name available somewhere
>> for <module> object. All right, this kind of tricky - '1 +1' a separate
>> module, right? It is anonymous and doesn't have a name. Still is it
>> possible to identify it somehow? For printing in function traces.
>>
>
> An exception traceback gives you the answer; with the script foo.py below:
>
> def f():
>   exec 'x + 1'
> f()
>
> I get:
>
> Traceback (most recent call last):
>   File "/tmp/foo.py", line 4, in <module>
>     f()
>   File "/tmp/foo.py", line 2, in f
>     exec 'x + 1'
>   File "<string>", line 1, in <module>
> NameError: name 'x' is not defined
>
> Do you see all the information you need here?
> If yes, it's possible to extract it from the current frames.
>


Not really. It got too far from original user story about execution
contexts, so I
need to choose a different entry point to continue discussion. What I need
is the
graph of state machine of Python execution. I want to inspect possible
starting
points, from which select which are visible to the user in which cases.

Still, thanks to the pointer. I fixed the .f_back exception in version 0.3
of xtrace,
and added a separate function - beep() that allows extending xtrace with
your
own stuff for inspecting execution process.

This module opened a Pandora box of Python internals. Version 0.4 still
fails to
trace files specified on command line, and I am lost in internal details of
execfile
+ locals()/globals()/namespacing/scoping. Python tracker doesn't help here.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130325/4034f373/attachment-0001.html>

From amauryfa at gmail.com  Mon Mar 25 18:13:43 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 25 Mar 2013 18:13:43 +0100
Subject: [pypy-dev] Python Execution Contexts
In-Reply-To: <CAPkN8xLMd-KjbXBuOrnjRipbKnfbVG9WSvzQNN33AiQoyH6ykg@mail.gmail.com>
References: <CAPkN8x+2qoTJhcbH_XLyaELWRVA5MGxp_gj6EFsY5Tq2h8WPkQ@mail.gmail.com>
	<CAGmFidZkbRzPOtm6VB7-Db5c1TcY+8nTcKOpzZr2gLwZWaajWw@mail.gmail.com>
	<CAPkN8x+=4Q=9_Q0YddfMkvT7sa-2Wij5JKOZrWx=mwevA=giYg@mail.gmail.com>
	<CAGmFidZ_Ei6WO34M12zBX3XsvaeEMa8sBnUn=pHYkLDnOioYXg@mail.gmail.com>
	<CAPkN8xJvc08rVnEzuj7XCbR20uvk4wx=M1Qy+UuwidpC-ZZWPg@mail.gmail.com>
	<CAGmFidaQ2ja0rd=Ze7Lx03uQ7UofPXWDt5j0_4QqEpC81xSheQ@mail.gmail.com>
	<CAPkN8xLMd-KjbXBuOrnjRipbKnfbVG9WSvzQNN33AiQoyH6ykg@mail.gmail.com>
Message-ID: <CAGmFidYvRUaeyxXvzpS31aFwLkoqnq5=zsjhLeSSsU06R6nn=w@mail.gmail.com>

2013/3/25 anatoly techtonik <techtonik at gmail.com>

> This module opened a Pandora box of Python internals. Version 0.4 still
> fails to
> trace files specified on command line, and I am lost in internal details
> of execfile
> + locals()/globals()/namespacing/scoping. Python tracker doesn't help here.
>

Seriously: always always pass your own locals and globals to functions like
exec and execfile.
Good luck.


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130325/4923e387/attachment.html>

From techtonik at gmail.com  Tue Mar 26 17:55:15 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Tue, 26 Mar 2013 19:55:15 +0300
Subject: [pypy-dev] Python Execution Contexts Ch.2
Message-ID: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>

On Mon, Mar 25, 2013 at 8:13 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

>
> 2013/3/25 anatoly techtonik <techtonik at gmail.com>
>
>> This module opened a Pandora box of Python internals. Version 0.4 still
>> fails to
>> trace files specified on command line, and I am lost in internal details
>> of execfile
>> + locals()/globals()/namespacing/scoping. Python tracker doesn't help
>> here.
>>
>
> Seriously: always always pass your own locals and globals to functions
> like exec and execfile.
> Good luck.


I'd like to understand what's going on there, because it directly affects
if I will be able to extend xtrace further and troubleshoot reported
problems when more errors appear.

I know that locals() is not an ordinary dictionary (
http://bugs.python.org/issue17546). And the first question - what should I
pass to execfile as 'my own locals'?
1. an exact livedict that is returned by locals() function
2. my own dict, which will become live inside the execfile call
3. locals function that returns a reference to livedict


The second question - why should I do this?

To make the explanation easier, here is a real world problem that made me
seek the answers.
https://bitbucket.org/techtonik/xtrace/issue/3/unable-to-xtrace-indefpy

The file that execfile() *inside* xtrace fails to execute is below. I named
it indef.py, because subprocess call is inside defined function.

import subprocess
def ret():
  print subprocess.check_output(['hg', 'id', '-nib'])
ret()


The problem repeats if the file is executed from console, but only in
special case:

    Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
(Intel)]
    >>> execfile('indef.py')
    0cef574f7f62+ 23+ default

    >>> def x():
    ...   execfile('indef.py')
    ...
    >>> x()
    0cef574f7f62+ 23+ default

Now I restart the console and do the same stuff, but without the first call:

    Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit
(Intel)]
    >>> def x():
    ...   execfile('indef.py')
    ...
    >>> x()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<stdin>", line 2, in x
      File "indef.py", line 6, in <module>
        ret()
      File "indef.py", line 4, in ret
        print subprocess.check_output(['hg', 'id', '-nib'])
    NameError: global name 'subprocess' is not defined

I guess the same stuff happens with my xtrace(), where execfile() is
executed inside the function. Now the questions:

1. I understand that when Python starts executing code, it creates a
namespace. This is the 'first_namespace' where Python starts to store all
variables encountered. I intentionally do not call it global, as this term
is not introduced at this point. Python continues to put and change
variables there until it encounters [something], this [something] causes it
to create a 'new_namespace' to store variables. I know that a function call
makes Python create 'new_namespace' to store variables encountered inside
this function, so "function call" is in [something]. What else is in
[something], i.e. what else causes Python to create 'new_namespaces'? It
will make me comfortable to see the full list described somewhere.

2. 'new_namespace's are created when function is entered and destroyed when
it exits. When variable is requested, the lookup mechanism looks first
inside 'new_namespace', then checks its parent (caller namespace), then
parent of the parent and so on until the 'first_namespace'. Does it work
like this?

3. Now the problem. When I run execfile() without parameters, I expect the
code inside to be 'virtualized' - isolated from parent process, like in LXC
or VirtualBox, but on Python level. I also expect that if I want to
communicate with this code inside execfile(), I craft my own
'first_namespace' and pass it down. When execfile() returns, I inspect the
state of this modified namespace, and maybe import some results back into
my 'current_namespace'. The example with indef.py above shows the
execfile() doesn't work this way. So the question is - why execfile()
doesn't it work the way I described?

4. At this point I hope to reach the state there it is clear how execfile()
works in reality, and the next question in this state - what was the reason
to make it so complicated? Optimization?

Thanks for the feedback.
-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130326/14616819/attachment.html>

From amauryfa at gmail.com  Tue Mar 26 19:26:30 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 26 Mar 2013 19:26:30 +0100
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
Message-ID: <CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>

2013/3/26 anatoly techtonik <techtonik at gmail.com>

> 3. Now the problem. When I run execfile() without parameters, I expect the
> code inside to be 'virtualized' - isolated from parent process, like in LXC
> or VirtualBox, but on Python level. I also expect that if I want to
> communicate with this code inside execfile(), I craft my own
> 'first_namespace' and pass it down. When execfile() returns, I inspect the
> state of this modified namespace, and maybe import some results back into
> my 'current_namespace'. The example with indef.py above shows the
> execfile() doesn't work this way. So the question is - why execfile()
> doesn't it work the way I described?
>

You should always pass locals and globals to execfile(). I warned you :-)

No, the code is not isolated from the parent. When you call execfile only
with a filename, globals is set tho the caller's globals, and locals are
set to the caller's locals!


> 4. At this point I hope to reach the state there it is clear how
> execfile() works in reality, and the next question in this state - what was
> the reason to make it so complicated? Optimization?
>

OK, here is an explanation of what you probably call the "Python Execution
Context":

- All Python code runs with two namespaces: locals and globals.
- When running a script, or when a module is imported, there is only one
namespace: locals() is globals().
- When a function object is created "def...", the globals namespace is
captured, and stored in the function object.
  When the function is executed, a new locals namespace is created, and
filled with the function arguments.
  The function bytecode is executed with these locals; globals is the one
captured above.
- [Missing: the class statement]

When running bytecode:
- All assignments go into the locals namespace (this includes "x+=1", but
also "import sys")
- Name lookups first search locals, then globals.
- Except inside a function scope ("def", or a lambda), where the compiler
determines whether the name is a local variable
  (it has an assignment in the same scope), in this case only the locals
are searched; otherwise only globals are searched.
  [For this discussion, it's only an implementation detail and makes no
difference here]
- [Also missing: nested scopes and cells]

I think this explains all the weirdness you see:
- "import subprocess" is an assignment, and will populate the local
namespace.
- when ret() sees the "subprocess" name, it will lookup the globals
namespace.
In a normal script/module, this makes no difference, because locals() is
globals().

But with execfile(), globals and locals are passed from the parent frame:
- when execfile() is called at module level, they are identical: locals()
is globals() in the caller.
- when execfile() is called inside a function, they are different! And boom.


In other words, follow the advice above:
    namespace = {}
    execfile('indef.py', namespace)
This will pass the same namespace for both globals and locals,
and indef.py will be executed a bit like "import" or a standalone script.


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130326/b1bda478/attachment.html>

From skip at pobox.com  Tue Mar 26 20:27:43 2013
From: skip at pobox.com (Skip Montanaro)
Date: Tue, 26 Mar 2013 14:27:43 -0500
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
	<CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
Message-ID: <CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>

>> 3. Now the problem. When I run execfile() without parameters, I expect the
>> code inside to be 'virtualized' - isolated from parent process, like in LXC
>> or VirtualBox, but on Python level.

> No, the code is not isolated from the parent. When you call execfile only
> with a filename, globals is set tho the caller's globals, and locals are set
> to the caller's locals!

Though this is just an educated guess, reading Anatoly's note and
Amaury's response leads me to think that Anatoly might be expecting
the execfile() function to work more or less like the exec* system
calls on Unix systems.  Execfile() is more akin to the import
statement than the fork/exec paradigm.

Skip

From amauryfa at gmail.com  Tue Mar 26 21:31:48 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 26 Mar 2013 21:31:48 +0100
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
	<CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
	<CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>
Message-ID: <CAGmFidYRCa+SUB4RE8ZRNuy3xXbM1VSn-j0iuXygdh=CG02XXw@mail.gmail.com>

Amaury Forgeot d'Arc
Le 26 mars 2013 20:27, "Skip Montanaro" <skip at pobox.com> a ?crit :
>
> >> 3. Now the problem. When I run execfile() without parameters, I expect
the
> >> code inside to be 'virtualized' - isolated from parent process, like
in LXC
> >> or VirtualBox, but on Python level.
>
> > No, the code is not isolated from the parent. When you call execfile
only
> > with a filename, globals is set tho the caller's globals, and locals
are set
> > to the caller's locals!
>
> Though this is just an educated guess, reading Anatoly's note and
> Amaury's response leads me to think that Anatoly might be expecting
> the execfile() function to work more or less like the exec* system
> calls on Unix systems.  Execfile() is more akin to the import
> statement than the fork/exec paradigm.

But at least import runs code in a fresh namespace; execfile does not.

>
> Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130326/55fe0a52/attachment.html>

From arigo at tunes.org  Tue Mar 26 23:00:34 2013
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 26 Mar 2013 23:00:34 +0100
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CAGmFidYRCa+SUB4RE8ZRNuy3xXbM1VSn-j0iuXygdh=CG02XXw@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
	<CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
	<CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>
	<CAGmFidYRCa+SUB4RE8ZRNuy3xXbM1VSn-j0iuXygdh=CG02XXw@mail.gmail.com>
Message-ID: <CAMSv6X05gKKaQDJXSb=hxgUHayWJe2CEu7AT-b+NaNW0THPYHg@mail.gmail.com>

Hi all,

Could you please move this debate somewhere else?  pypy-dev is not the
appropriate place where to discuss the Python language in general (but
topics about differences between CPython and PyPy are fine of course).
 There is already for example
http://mail.python.org/mailman/listinfo/python-list .


A bient?t,

Armin.

From techtonik at gmail.com  Wed Mar 27 16:05:38 2013
From: techtonik at gmail.com (anatoly techtonik)
Date: Wed, 27 Mar 2013 18:05:38 +0300
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CAMSv6X05gKKaQDJXSb=hxgUHayWJe2CEu7AT-b+NaNW0THPYHg@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
	<CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
	<CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>
	<CAGmFidYRCa+SUB4RE8ZRNuy3xXbM1VSn-j0iuXygdh=CG02XXw@mail.gmail.com>
	<CAMSv6X05gKKaQDJXSb=hxgUHayWJe2CEu7AT-b+NaNW0THPYHg@mail.gmail.com>
Message-ID: <CAPkN8xLALehg6w1HAnCCZ6756_Jhd62vDJsB6zcs_jVPTRK5Nw@mail.gmail.com>

First things first. Thanks for replies. I am still crunching..

As I already received a similar question about if the topic is appropriate
for PyPy in private email from other person, so I repeat my reply below
as-is. Mind you the the discussion here is not "debate". When I say "I
expect" - I of course mention my own problem of understanding, but the
answer I need is not what should I do, but why it works differently? I am
not familiar with exec* calls in Unix, but for me the analogy is
misleading, and something like "LXC for Python code" may be a better
analogy, because I don't want to replace the current process. What I am
interested to know is what kind of code isolation execfile() supports and
why it doesn't do full isolation by default? It looks like this is
answered, but I need time to validate it.

On Wed, Mar 27, 2013 at 1:00 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> Could you please move this debate somewhere else?  pypy-dev is not the
> appropriate place where to discuss the Python language in general (but
> topics about differences between CPython and PyPy are fine of course).
>  There is already for example
> http://mail.python.org/mailman/listinfo/python-list .
>


It looks like python@ is more appropriate for basic questions about the
usage of the existing language. My question is about internal mechanism
that supports language design. Not only asking about existing behavior, but
also about why this behavior can not be changed. I thought that people who
are writing Python in Python are more skilled to answer such questions. In
the end, I understood that PyPy project was created so that people who
can't read C could still understand and experiment with their own language,
no?

-- 
anatoly t.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130327/aec77bda/attachment.html>

From arigo at tunes.org  Wed Mar 27 19:04:02 2013
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Mar 2013 19:04:02 +0100
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
Message-ID: <CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>

Hi Sebastien,

On Mon, Mar 25, 2013 at 7:28 AM, Sebastien Douche <sdouche at gmail.com> wrote:
> How is the state of the STM branch? Just a POC or Armin has a idea for
> producing a new version of PyPy?

What is "POC"?

I have certainly more than an idea: there is a working prototype.
It's missing some things as described on a previous mail to pypy-dev:
http://mail.python.org/pipermail/pypy-dev/2013-February/010900.html


A bient?t,

Armin.

From sparks.m at gmail.com  Wed Mar 27 19:23:05 2013
From: sparks.m at gmail.com (Michael)
Date: Wed, 27 Mar 2013 18:23:05 +0000
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
	<CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
Message-ID: <CAB+QZVAy5jG29TqWG=cOL5ET-G7B+1f6gdjHi0kdx3iu5BbZ=Q@mail.gmail.com>

Hi,

I'd assume POC means "proof of concept".

Michael.

On 27/03/2013, Armin Rigo <arigo at tunes.org> wrote:
> Hi Sebastien,
>
> On Mon, Mar 25, 2013 at 7:28 AM, Sebastien Douche <sdouche at gmail.com>
> wrote:
>> How is the state of the STM branch? Just a POC or Armin has a idea for
>> producing a new version of PyPy?
>
> What is "POC"?
>
> I have certainly more than an idea: there is a working prototype.
> It's missing some things as described on a previous mail to pypy-dev:
> http://mail.python.org/pipermail/pypy-dev/2013-February/010900.html
>
>
> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

From skip at pobox.com  Wed Mar 27 19:28:59 2013
From: skip at pobox.com (Skip Montanaro)
Date: Wed, 27 Mar 2013 13:28:59 -0500
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAB+QZVAy5jG29TqWG=cOL5ET-G7B+1f6gdjHi0kdx3iu5BbZ=Q@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
	<CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
	<CAB+QZVAy5jG29TqWG=cOL5ET-G7B+1f6gdjHi0kdx3iu5BbZ=Q@mail.gmail.com>
Message-ID: <CANc-5UzKCkm0d1OMe1R=KmJ6+wG8fNwW5r6HgJMyrz+HUqwhHA@mail.gmail.com>

> I'd assume POC means "proof of concept".

I hope it doesn't mean the other expansion which jumped
(involuntarily) to mind when I read it!

Skip

From sparks.m at gmail.com  Wed Mar 27 19:33:23 2013
From: sparks.m at gmail.com (Michael)
Date: Wed, 27 Mar 2013 18:33:23 +0000
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CANc-5UzKCkm0d1OMe1R=KmJ6+wG8fNwW5r6HgJMyrz+HUqwhHA@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
	<CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
	<CAB+QZVAy5jG29TqWG=cOL5ET-G7B+1f6gdjHi0kdx3iu5BbZ=Q@mail.gmail.com>
	<CANc-5UzKCkm0d1OMe1R=KmJ6+wG8fNwW5r6HgJMyrz+HUqwhHA@mail.gmail.com>
Message-ID: <CAB+QZVD2hBgRXLeAnpQqVVptGCAd_vhg355U6CZvjPf44caAtQ@mail.gmail.com>

Hi,

Have more faith! :)

Michael.

On 27/03/2013, Skip Montanaro <skip at pobox.com> wrote:
>> I'd assume POC means "proof of concept".
>
> I hope it doesn't mean the other expansion which jumped
> (involuntarily) to mind when I read it!
>
> Skip
>

From amauryfa at gmail.com  Thu Mar 28 01:09:25 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 28 Mar 2013 01:09:25 +0100
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
Message-ID: <CAGmFidYo=2dJufiVWghAeAtdR1dUwaH+Y=Dn6pO_zjxMiuf1zA@mail.gmail.com>

2013/3/23 Alex Gaynor <alex.gaynor at gmail.com>

> Hi everyone,
>
> I've started putting together an ideas page for this GSOC:
> https://bitbucket.org/pypy/pypy/wiki/GSOC%202013
>
> If you've got an idea, or are interested in mentoring you should add
> yourself/the idea.
>

I added myself as a potential mentor, and a project for a wxPython-on-cffi
port.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130328/9e7ceacd/attachment-0001.html>

From daniel.strauss8 at gmail.com  Thu Mar 28 00:44:50 2013
From: daniel.strauss8 at gmail.com (Daniel Strauss)
Date: Wed, 27 Mar 2013 16:44:50 -0700
Subject: [pypy-dev] Tips to Improve Your Memory
Message-ID: <FXVRYB0-4FQU-CMS-YFN-FUUQ5NO3P1L@gmail.com>

Hi,

I'm a contributor to an education improvement resource and I recently published an article providing many tips about improving your memory I think you'd be interested in. I found your site http://bugs.python.org/issue15490 during my research and would like to invite you to take a look at my published piece.

Please let me know if you'd be interested in taking a look and possibly sharing with your readers.

Thanks,

Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130327/d1b92fd9/attachment.html>

From fijall at gmail.com  Thu Mar 28 09:44:41 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 28 Mar 2013 01:44:41 -0700
Subject: [pypy-dev] Python Execution Contexts Ch.2
In-Reply-To: <CAPkN8xLALehg6w1HAnCCZ6756_Jhd62vDJsB6zcs_jVPTRK5Nw@mail.gmail.com>
References: <CAPkN8xKjGxdZz5xLy6+bD_Z_K3mBf5Pvuu--pAXX7tmzjUnkNA@mail.gmail.com>
	<CAGmFidbcJtCR7XXDJ+RmWn1uSEgTeMrt6DsNuU5xmuQxdOsOGA@mail.gmail.com>
	<CANc-5UyAZCbWdAkZBrnhVRhzg8eNeBxb=89mMD53_M-YCiRNbA@mail.gmail.com>
	<CAGmFidYRCa+SUB4RE8ZRNuy3xXbM1VSn-j0iuXygdh=CG02XXw@mail.gmail.com>
	<CAMSv6X05gKKaQDJXSb=hxgUHayWJe2CEu7AT-b+NaNW0THPYHg@mail.gmail.com>
	<CAPkN8xLALehg6w1HAnCCZ6756_Jhd62vDJsB6zcs_jVPTRK5Nw@mail.gmail.com>
Message-ID: <CAK5idxQ_-+Gf1r8ezU7fLB4Q6YmLeuSYu47S3TemOQ2eAWcFQg@mail.gmail.com>

> It looks like python@ is more appropriate for basic questions about the
> usage of the existing language. My question is about internal mechanism that
> supports language design. Not only asking about existing behavior, but also
> about why this behavior can not be changed. I thought that people who are
> writing Python in Python are more skilled to answer such questions. In the
> end, I understood that PyPy project was created so that people who can't
> read C could still understand and experiment with their own language, no?

We can explain to you how stuff works. Why not change it? Because
python-dev said "no". Please take it with them, we don't deal with
language design. People *can* experiment with the language whatever
they like. If they find their experiments working, they can take
python changes with python-dev, we'll not participate in this
discussion.

You're not asking how it's implemented though, you're asking why it
works the way it works, and this is totally not for us to answer - we
just copied cpython semantics and this is what we're going to keep.
Please take this discussion somewhere else.

From sdouche at gmail.com  Thu Mar 28 13:19:45 2013
From: sdouche at gmail.com (Sebastien Douche)
Date: Thu, 28 Mar 2013 13:19:45 +0100
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
	<CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
Message-ID: <CAAGHeXH_kyt2TmoGCWn=OV=R7fEZLWB6QXAi9qOmRi3uJKVN-w@mail.gmail.com>

On Wed, Mar 27, 2013 at 7:04 PM, Armin Rigo <arigo at tunes.org> wrote:
>> How is the state of the STM branch? Just a POC or Armin has a idea for
>> producing a new version of PyPy?
>
> What is "POC"?

Sorry, "proof of concept". I surprised to not see STM in the list.


--
Sebastien Douche <sdouche at gmail.com>
Twitter: @sdouche / G+: +sdouche

From arigo at tunes.org  Thu Mar 28 16:50:45 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Mar 2013 16:50:45 +0100
Subject: [pypy-dev] GSOC 2013
In-Reply-To: <CAAGHeXH_kyt2TmoGCWn=OV=R7fEZLWB6QXAi9qOmRi3uJKVN-w@mail.gmail.com>
References: <CAFRnB2WQAGMYcUoEmWLKSsQqG3rQ8y1_=u8DQP4CXNsXhNdgZQ@mail.gmail.com>
	<CAAGHeXHK3aPFFmW0Ln-xEa_4V0ben_UpygVh3yR1jvroOQaLdg@mail.gmail.com>
	<CAMSv6X34bGgQ8YixMZFW+diwjrzwLoer7VV_Kr2vDXYdG1P+gg@mail.gmail.com>
	<CAAGHeXH_kyt2TmoGCWn=OV=R7fEZLWB6QXAi9qOmRi3uJKVN-w@mail.gmail.com>
Message-ID: <CAMSv6X2=WB7EZTN_zCDL-y8yF-B1PHA98r+8Zujjb=MUQHmvrw@mail.gmail.com>

Hi Sebastien,

On Thu, Mar 28, 2013 at 1:19 PM, Sebastien Douche <sdouche at gmail.com> wrote:
> Sorry, "proof of concept". I surprised to not see STM in the list.

Ah.  I don't think it's very well-suited for GSoC.  I added a
paragraph at the end of
https://bitbucket.org/pypy/pypy/wiki/GSOC%202013 .


A bient?t,

Armin.

From arigo at tunes.org  Thu Mar 28 17:33:55 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Mar 2013 17:33:55 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: use
 jit.loop_unrolling_heuristic where possible
In-Reply-To: <5150426E.3090906@gmx.de>
References: <20130325094239.C28781C00E4@cobra.cs.uni-duesseldorf.de>
	<5150426E.3090906@gmx.de>
Message-ID: <CAMSv6X0RDnSQZYU8kkAMBG1uHBYMOrft_1CWU9vNQBDC57xRgg@mail.gmail.com>

Hi Brian, hi Carl Friedrich,

On Mon, Mar 25, 2013 at 1:26 PM, Carl Friedrich Bolz <cfbolz at gmx.de> wrote:
> m = l0
> if l1 < m:
>     m = l1
> if l2 < m
>     m = l2
> if l3 < m
>     m = l3
> if l3 < m
>     m = l3

Alternatively, we could work a bit more and have it jitted as a single
non-branching piece of code.  For example, with "space.min(w_a, w_b)",
which would check the types, and if both are ints, then just return
"space.newint(min(w_a.value, w_b.value))".  There are various way to
compute min(x, y) with two integer arguments in RPython, without
branching.  At worst we make it a special ResOperation, but there are
also other ways.


A bient?t,

Armin.

From arigo at tunes.org  Thu Mar 28 18:05:03 2013
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 28 Mar 2013 18:05:03 +0100
Subject: [pypy-dev] [pypy-commit] pypy default: use
 jit.loop_unrolling_heuristic where possible
In-Reply-To: <CAMSv6X0RDnSQZYU8kkAMBG1uHBYMOrft_1CWU9vNQBDC57xRgg@mail.gmail.com>
References: <20130325094239.C28781C00E4@cobra.cs.uni-duesseldorf.de>
	<5150426E.3090906@gmx.de>
	<CAMSv6X0RDnSQZYU8kkAMBG1uHBYMOrft_1CWU9vNQBDC57xRgg@mail.gmail.com>
Message-ID: <CAMSv6X0zZaX5ugjFUhbBo6We-KJYKOy3LKBk8=RSNrjvWoTU6A@mail.gmail.com>

Hi,

On Thu, Mar 28, 2013 at 5:33 PM, Armin Rigo <arigo at tunes.org> wrote:
> There are various way to compute min(x, y) with two integer
> arguments in RPython, without branching.

The best might be to implement min(x, y) in RPython as returning
"int_select(x < y, x, y)", using a new rarithmetic.int_select(a, b, c)
function similar to rarithmetic.int_between(), which would turn into
the C code "a ? b : c".  We can easily support this operation a bit
everywhere, down to the JIT backends --- and then on x86 it is done
with a single conditional move instruction.


A bient?t,

Armin.

From dem at smartbusinesscoach.it  Fri Mar 29 15:04:29 2013
From: dem at smartbusinesscoach.it (Smart Business Coach)
Date: Fri, 29 Mar 2013 15:04:29 +0100
Subject: [pypy-dev] Invito Esclusivo per l'IMPRENDITORE
Message-ID: <ODgxODc2MgAC407540Y14BAMTM2NDU2NTg2OTM3MDkz@www.smartbusinesscoach.it>

Caro Imprenditore,

Questa mail ti raggiunge per 
INVITARTI ad un EVENTO ESCLUSIVO
DEDICATO AI SOLI IMPRENDITORI
che abbiamo organizzato a Roma il 13 Aprile 2013,
DEDICATO A TE e alla TUA AZIENDA.

--------- CLICCA QUI ( http://www.smartbusinesslab.com/evento ) --------- 
per scoprire di cosa si tratta.

PS: Non ci conosci (Ancora) e ricevi 
questa mail perch? ABBIAMO ACQUISTATO 
da liste pubbliche il tuo contatto. 
Se vuoi, puoi disiscriverti da questa lista 
cliccando qui (
http://www.smartbusinesscoach.it/dem/index.php?subid=407540&option=com_acymailing&ctrl=user&task=out&mailid=14&key=202c460b0eb1b2ecf7e4cb62d095320c );


per cortesia visiona prima l'invito.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130329/21a77cf3/attachment.html>

From wlavrijsen at lbl.gov  Fri Mar 29 21:14:14 2013
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Fri, 29 Mar 2013 13:14:14 -0700 (PDT)
Subject: [pypy-dev] prevent extension module code from being executed during
	startup?
Message-ID: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>

Hi,

when starting pypy-c, I get some extension module code executed before that
module is imported.

The code in question is in pypy/module/cppyy/pythonify.py:

gbl = make_cppnamespace(None, "::", None, False)   # global C++ namespace
gbl.__doc__ = "Global C++ namespace."
sys.modules['cppyy.gbl'] = gbl

This code is not rpython code. The module is first mentioned in the
__init__.py of cppyy:

     appleveldefs = {
         'gbl'                    : 'pythonify.gbl',
         'load_reflection_info'   : 'pythonify.load_reflection_info',
         'add_pythonization'      : 'pythonify.add_pythonization',
     }

How can I prevent that code from being executed at startup, i.e. have it
deferred all the way until "import cppyy"?

The use case is as follows: I have a prototype where I load the Reflex
library (and associated C-API) at run-time, using _cffi_backend. That way,
it is no longer necessary to link pypy-c with libReflex.so. I can handle a
missing .so silently, but would prefer to raise an ImportError. However,
for that to work, I need to make sure nothing of the module gets touched
until it's imported, so that the error is reported at the right time.

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From arigo at tunes.org  Fri Mar 29 21:31:49 2013
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Mar 2013 21:31:49 +0100
Subject: [pypy-dev] prevent extension module code from being executed
 during startup?
In-Reply-To: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
References: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
Message-ID: <CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>

Hi Wim,

On Fri, Mar 29, 2013 at 9:14 PM,  <wlavrijsen at lbl.gov> wrote:
> The code in question is in pypy/module/cppyy/pythonify.py:
>
> gbl = make_cppnamespace(None, "::", None, False)   # global C++ namespace
> gbl.__doc__ = "Global C++ namespace."
> sys.modules['cppyy.gbl'] = gbl

Indeed, this runs ahead of time --- maybe already at translation time,
actually.  You can move this logic into a function _initialize_gbl(),
and remove the appleveldefs entry for 'gbl'.  Then call this new
function at import time, by adding a method like this in the class
Module in __init__.py:

def startup(self, space):
    space.call_method(space.wrap(self), "_initialize_gbl")


A bient?t,

Armin.

From wlavrijsen at lbl.gov  Sat Mar 30 07:08:51 2013
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Fri, 29 Mar 2013 23:08:51 -0700 (PDT)
Subject: [pypy-dev] prevent extension module code from being executed
 during startup?
In-Reply-To: <CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>
References: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
	<CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1303292302060.21230@pc3a-wlav.lbl.gov>

Hi Armin,

> Indeed, this runs ahead of time --- maybe already at translation time,
> actually.

yes, but that's fine: the code has no side effects. If it does not run at
translation time, the value of "gbl" seems to be frozen at its initial one,
so that's not an option.

> You can move this logic into a function _initialize_gbl(),
> and remove the appleveldefs entry for 'gbl'.  Then call this new
> function at import time, by adding a method like this in the class
> Module in __init__.py:
>
> def startup(self, space):
>    space.call_method(space.wrap(self), "_initialize_gbl")

That actually doesn't work: the translator complains about the setup of
applevel objects being circular (don't have the exact error message handy).
I've tried a range of variations, but no luck to get the desired outcome.

But anyway after I decided to leave gbl as-is, and simply put the check in
startup(), I found that it is run at program load, not on import, so that
doesn't help in the first place.

My current thinking is to weed the initialization down to the absolute
minimum (that's a good thing regardless :) ), then fail on first use if the
library is not available. Not ideal, but I guess in practice it makes little
or no difference.

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From _kfj at yahoo.com  Sat Mar 30 09:56:48 2013
From: _kfj at yahoo.com (Kay F. Jahnke)
Date: Sat, 30 Mar 2013 14:26:48 +0530
Subject: [pypy-dev] numpypy problem - can't find augmented assignment methods
Message-ID: <5156A8D0.5040408@yahoo.com>

Hi group!

I'm running numpypy and I have a problem with augmented assignment 
statements. It looks to me like the relevant methods are simply absent 
from my numpypy array objects. Consider for example:

$ pypy
Python 2.7.3 (7e4f0faa3d51, Nov 16 2012, 16:56:51)
[PyPy 2.0.0-beta1 with GCC 4.4.3] on linux2
...
 >>>> import numpypy as np
 >>>> a=np.array([1])
 >>>> b = a
 >>>> b is a
True
 >>>> b *= 3
 >>>> b is a
False
 >>>> b.__imul__
Traceback (most recent call last):
   File "<console>", line 1, in <module>
AttributeError: 'ndarray' object has no attribute '__imul__'

On numpy under CPython, 'b is a' evaluates to True, and 
np.ndarray.__imul__ is present. I suppose since the method isn't there, 
the system is falling back to run __mul__ and assigns the result to a 
new object (after all b contains correct data after the *=), but for my 
application I want the in-place operation. I've browsed in circles for a 
while to find out what's wrong or, respectively, why it is as it is; 
there might be a simple answer to my problem, so I decided to ask for 
help :)

With regards

Kay




From fijall at gmail.com  Sat Mar 30 11:55:15 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 30 Mar 2013 03:55:15 -0700
Subject: [pypy-dev] numpypy problem - can't find augmented assignment
	methods
In-Reply-To: <5156A8D0.5040408@yahoo.com>
References: <5156A8D0.5040408@yahoo.com>
Message-ID: <CAK5idxTfe6aqpJS=Mr9qoQHTqShUr8G1vKp+1Cp7WNvz3qq8Dg@mail.gmail.com>

On Sat, Mar 30, 2013 at 1:56 AM, Kay F. Jahnke <_kfj at yahoo.com> wrote:
> Hi group!
>
> I'm running numpypy and I have a problem with augmented assignment
> statements. It looks to me like the relevant methods are simply absent from
> my numpypy array objects. Consider for example:
>
> $ pypy
> Python 2.7.3 (7e4f0faa3d51, Nov 16 2012, 16:56:51)
> [PyPy 2.0.0-beta1 with GCC 4.4.3] on linux2
> ...
>>>>> import numpypy as np
>>>>> a=np.array([1])
>>>>> b = a
>>>>> b is a
> True
>>>>> b *= 3
>>>>> b is a
> False
>>>>> b.__imul__
> Traceback (most recent call last):
>   File "<console>", line 1, in <module>
> AttributeError: 'ndarray' object has no attribute '__imul__'
>
> On numpy under CPython, 'b is a' evaluates to True, and np.ndarray.__imul__
> is present. I suppose since the method isn't there, the system is falling
> back to run __mul__ and assigns the result to a new object (after all b
> contains correct data after the *=), but for my application I want the
> in-place operation. I've browsed in circles for a while to find out what's
> wrong or, respectively, why it is as it is; there might be a simple answer
> to my problem, so I decided to ask for help :)
>
> With regards
>
> Kay

It's simply unsupported by now :)

From arigo at tunes.org  Sat Mar 30 12:56:56 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Mar 2013 12:56:56 +0100
Subject: [pypy-dev] prevent extension module code from being executed
 during startup?
In-Reply-To: <alpine.LNX.2.00.1303292302060.21230@pc3a-wlav.lbl.gov>
References: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
	<CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>
	<alpine.LNX.2.00.1303292302060.21230@pc3a-wlav.lbl.gov>
Message-ID: <CAMSv6X3XmeDVnqCqAH-wnAajhyMKcw8rOSm9sbEirYRq3xOkmg@mail.gmail.com>

Hi Wim,

On Sat, Mar 30, 2013 at 7:08 AM,  <wlavrijsen at lbl.gov> wrote:
> That actually doesn't work: the translator complains about the setup of
> applevel objects being circular (don't have the exact error message handy).
> I've tried a range of variations, but no luck to get the desired outcome.

At this point, you need to provide more information (as I can't run
the tests myself).  Fwiw the patch I have in mind would look like
this: http://bpaste.net/show/87755/ and I have no clue about which
part of the code contains an error message about being circular.


A bient?t,

Armin.

From wlavrijsen at lbl.gov  Sat Mar 30 17:40:40 2013
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Sat, 30 Mar 2013 09:40:40 -0700 (PDT)
Subject: [pypy-dev] prevent extension module code from being executed
 during startup?
In-Reply-To: <CAMSv6X3XmeDVnqCqAH-wnAajhyMKcw8rOSm9sbEirYRq3xOkmg@mail.gmail.com>
References: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
	<CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>
	<alpine.LNX.2.00.1303292302060.21230@pc3a-wlav.lbl.gov>
	<CAMSv6X3XmeDVnqCqAH-wnAajhyMKcw8rOSm9sbEirYRq3xOkmg@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1303300927450.24503@pc3a-wlav.lbl.gov>

Hi Armin,

> Fwiw the patch I have in mind would look like
> this: http://bpaste.net/show/87755/

this line:

-        'gbl'                    : 'pythonify.gbl',

can not be removed, as w/o it 'gbl' will not show up at the module level.

>  and I have no clue about which
> part of the code contains an error message about being circular.

Looks like this (and I get it when running the translation with the patch
as above, but with 'gbl' still in the applevel defs). Sorry, the message
said recursive, not circular:

[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/interpreter/mixedmodule.py", line 81, in getdictvalue
[translation:ERROR]     return self._load_lazily(space, name)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/interpreter/mixedmodule.py", line 91, in _load_lazily
[translation:ERROR]     w_value = loader(space)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/interpreter/mixedmodule.py", line 245, in afileloader
[translation:ERROR]     return app.wget(space, attrname)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/interpreter/gateway.py", line 999, in wget
[translation:ERROR]     w_globals = self.getwdict(space)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/pypy/interpreter/gateway.py", line 992, in getwdict
[translation:ERROR]     return space.fromcache(ApplevelCache).getorbuild(self)
[translation:ERROR]    File "/home/wlav/pypy-dev/pypy/rpython/rlib/cache.py", line 48, in getorbuild
[translation:ERROR]     self, key)
[translation:ERROR]  Exception: ("<Cache 'ApplevelCache' (18 items)> recursive building of <ApplevelClass filename='/home/wlav/pypy-dev/pypy/pypy/module/cppyy/pythonify.py'>", <
[translation:ERROR]  v42 = getattr(self_2, ('builtin_modules'))
[translation:ERROR] In <FunctionGraph of (pypy.interpreter.baseobjspace:407)getbuiltinmodule at 0x2fe303a0>:
[translation:ERROR] Happened at file /home/wlav/pypy-dev/pypy/pypy/interpreter/baseobjspace.py line 422
[translation:ERROR]
[translation:ERROR] ==>             w_mod = self.builtin_modules[name]
[translation:ERROR]
[translation:ERROR] Known variable annotations:
[translation:ERROR]  self_2 = SomePBC(can_be_None=False, const=StdObjSpace, subset_of=None)>)
[translation:ERROR] Processing block:
[translation:ERROR]  block at 120 is a <class 'rpython.flowspace.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.interpreter.baseobjspace:407)getbuiltinmodule
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v42 = getattr(self_2, ('builtin_modules'))
[translation:ERROR]        v43 = getitem_key(v42, name_0)
[translation:ERROR]  --end--

But again, the real problem is that startup() still runs immediately when I
type './pypy-c' and not when 'import cppyy' later on the prompt, so even if
the above problem gets solved, I still can't raise an ImportError in startup()
when the library is missing.

That is, if I leave the setup as-is and simply call verify_backend() in the
startup() function in cppyy/__init__.py, this is how it looks if the library
is missing:

  $ ./pypy-c
  Traceback (most recent call last):
    File "app_main.py", line 52, in run_toplevel
  ImportError: missing reflection module rflxlib.so!
  Python 2.7.3 (2377fb34943d+3ef424d4281f+, Mar 30 2013, 03:53:47)
  [PyPy 2.0.0-beta1 with GCC 4.6.2] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  And now for something completely different: ``things worked nicely till today
  just by chance''
  >>>>

(And copying rflxlib.so to an accessible location nicely makes the error
message go away, so the checking does work properly, unless that message I
got on the prompt is being prophetic. :) )

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From arigo at tunes.org  Sat Mar 30 18:25:11 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Mar 2013 18:25:11 +0100
Subject: [pypy-dev] prevent extension module code from being executed
 during startup?
In-Reply-To: <alpine.LNX.2.00.1303300927450.24503@pc3a-wlav.lbl.gov>
References: <alpine.LNX.2.00.1303291305570.9173@pc3a-wlav.lbl.gov>
	<CAMSv6X07rCav4N92ddEcX9t3cBu_g-Nft=vfto6RDXvX7Ann-Q@mail.gmail.com>
	<alpine.LNX.2.00.1303292302060.21230@pc3a-wlav.lbl.gov>
	<CAMSv6X3XmeDVnqCqAH-wnAajhyMKcw8rOSm9sbEirYRq3xOkmg@mail.gmail.com>
	<alpine.LNX.2.00.1303300927450.24503@pc3a-wlav.lbl.gov>
Message-ID: <CAMSv6X2D1O1Oj7JGs2viZU4BZ20-bYX8MDFmajbbdV98R8Yv-Q@mail.gmail.com>

Hi Wim,

On Sat, Mar 30, 2013 at 5:40 PM,  <wlavrijsen at lbl.gov> wrote:
> -        'gbl'                    : 'pythonify.gbl',
> can not be removed, as w/o it 'gbl' will not show up at the module level.

Ah I see, pythonify.py does "sys.modules['cppyy.gbl'] = gbl".  I
somehow misread that.  It should also do in the same place "cppyy.gbl
= gbl" to force the value to show up when this initialization code
runs.

> But again, the real problem is that startup() still runs immediately when I
> type './pypy-c' and not when 'import cppyy' later on the prompt

Uh, that's strange.  The docstrings in interpreter/module.py say
specifically the opposite.  But the truth looks a bit more complicated
indeed, e.g. it depends if getbuiltinmodule('cppyy') was already
called during translation or not...


A bient?t,

Armin.