From faassen at startifact.com  Fri Jun  1 10:06:38 2012
From: faassen at startifact.com (Martijn Faassen)
Date: Fri, 1 Jun 2012 10:06:38 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <20120531211809.GP10174@merlinux.eu>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
Message-ID: <CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>

Hi there,

On Thu, May 31, 2012 at 11:18 PM, holger krekel <holger at merlinux.eu> wrote:
> ?After all, this is only an intermediate solution until
> everthing happily runs on Python3 anyway, right? ;)

Well, I'm more thinking about it as a means to get to such a
situation, not as an intermediate solution. If you have an integrated
Python 2 and Python 3 in a single project you can decide to port code
to Python 3 on a per-module basis instead of on a per-project basis.

So I'm not any immediate need to run Python 3 libraries that I need to
solve. Deciding to do so for existing projects at this point would
cause quite a lot more hassle than I'm looking for. This is helping to
retard Python 3 adoption, as only new projects can start using it.

Concerning performance overhead of proxies, lifecycle issues would be
tricky, though starting with the basic notion that a proxy is like
another reference to that object in the interpreter would get you
quite far, I think? I do suspect a proxy approach could be made to be
very efficient.

I don't know enough about execnet in practice to know how it would
feel. It'd be nice if this were a documented way to integrate
different interpreters.

Regards,

Martijn

From stefano at rivera.za.net  Fri Jun  1 12:57:17 2012
From: stefano at rivera.za.net (Stefano Rivera)
Date: Fri, 1 Jun 2012 12:57:17 +0200
Subject: [pypy-dev] PyPy PPA
In-Reply-To: <CAFRnB2U1X3UtJXZ9Ev2dgt-FcLHzwb1UWYB+EGPWsCq_=HJKPQ@mail.gmail.com>
References: <CAFRnB2U1X3UtJXZ9Ev2dgt-FcLHzwb1UWYB+EGPWsCq_=HJKPQ@mail.gmail.com>
Message-ID: <20120601105717.GL1728@bach.rivera.co.za>

Hi Alex (2012.05.31_22:28:01_+0200)
> I'm hoping the people behind https://launchpad.net/~pypy/+archive/ppa are
> hear :)  I'm wondering what the status of this (or any other PPAs) are, and
> whether it'd be possible to get a PPA that works with Ubuntu 12.04.  The
> people behind http://travis-ci.org/ which is an awesome CI service, would
> like to be able to deploy PyPy via a deb on 12.04.

So, I maintain PyPy in Debian/Ubuntu itself. PyPy 1.8 is in 12.04, so
they can just install that, unless they are wanting newer releases as
they become available. I will probably backport future releases to
precise-backports, but won't commit to doing that for the life of the
release.

I was recently added to the team that manages that PPA, with the
intention of overhauling it, but haven't done anything about that yet.

The PPA's packaging is a bit weird and definitely needs an overhaul.
I'm intending to just use the same packaging approach as the distro
package (including the patches I'm applying, such as PEP3147 support)

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

From tbaldridge at gmail.com  Fri Jun  1 23:15:42 2012
From: tbaldridge at gmail.com (Timothy Baldridge)
Date: Fri, 1 Jun 2012 16:15:42 -0500
Subject: [pypy-dev] PyPy build bots?
Message-ID: <CAL36E+vkoEgUEzZKVpHXok1dONzyzRugfmxJ7dLZT1eAFrB_UA@mail.gmail.com>

I'm at the point where I'd like to start somewhat regular builds of my
pypy branch complete with JIT optimization. I tried downloading the
build bot but ran into some errors. Does pypy have a build farm that
branches can use? Any suggestions on the path-of-least resistance to
fire off nightly builds?

Thanks,

Timothy

From wlavrijsen at lbl.gov  Sat Jun  2 02:38:49 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Fri, 1 Jun 2012 17:38:49 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <1543846.T9Sm08LvAA@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1205301821010.1119@pc3-wlav.lbl.gov>
	<1543846.T9Sm08LvAA@hunter-laptop>
Message-ID: <alpine.LNX.2.00.1206011548200.1119@pc3-wlav.lbl.gov>

Hi Alex,

> Thanks! I have accumulated some understanding about the process, and the issue
> can be worked around by having a dictionary that would map ints into python
> objects and vice versa before those are sent to c++, so when c++ functions
> feth them back they can be retreived.

good you found a workaround, as using cpyext from cppyy is turning out to be a
bigger can of worms that I expected. :} Will take a tad bit longer: I have some
working code, but only post-translation. For some reason, the resulting objects
look nothing as expected in the test and as passed, they go into the CPython
interpreter (this is a test, after all), which then bombs.

Post-translation, it's looking fine, though, and working as expected.

Performance ... is absolutely deplorable: slower than PyROOT on CPython!

My guess is that in the micro-benchmark, the JIT gets no chance of making a
trace out of the loop iteration, since the loop iteration requires the PyObject
in every call (the PyObject that I'm passing is the iterator variable itself).
I'll have to look into more detail ....

However, if I do not pass the loop iterator, but pass a constant, it is very,
very zippy (though far from what can be done by unwrapping the constant). My
guess here is that the JIT can move the constant out of the loop iteration,
and this thus spares all those cpyext calls. Iow., it isn't actually fast,
but the JIT has its way with the loop. Again, I need to look into that.

So, I can't check in any code that doesn't pass a test, and since you'd have
to rebuild anyway, let's hold off with checking this code in (unless you are
happy with a patch?). I'll get to it soon enough.

> With respect to build, I installed root from gentoo repository, and the
> problem there is that headers are in /usr/include/root and the actual libs are
> in /usr/lib/root, but the pypy build expects libs to be in
> /usr/include/root/lib if you specify SYSROOT=/usr/include/root.

Oops. Yes, that is something that can be solved easily enough.

> With respect to genreflex command, you could probably warn people that they
> should use gcc 4.3, not 4.6, cuz gccxml, used by genreflex, is not compatible
> with newer versions of STL, and crashes on standard headers.

That shouldn't be ... on my box I have gcc 4.6.2 and that does not pose any
problems. What version of gccxml do you have? Mine's at 0.9.0.

> Also, any headers
> that are not absolutely necessary to define the interface to Python should be
> removed if possible, that does make life a lot easier, even with SWIG, because
> it makes wrappers smaller.

Have you looked into providing a selection.xml to genreflex to select the
needed classes? Also, another common option is to collect all desired headers
in a single header file (with simple #include's) and feed that to genreflex,
together with a selection.xml that simply uses "*" patterns.

> PS: If you want I could give you benchmark results of cppyy vs cpyext+SWIG
> when we get it running, probably it will be of some interest.

My benchmarks are pure C++. :) But yes, I'm interested in any result, although
for now, the binary benchmark "it works yes/no" is already interesting.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From nikuda at gmail.com  Sat Jun  2 04:24:14 2012
From: nikuda at gmail.com (ante)
Date: Sat, 2 Jun 2012 12:24:14 +1000
Subject: [pypy-dev] New pypy website update
Message-ID: <CAPPs_vCyd1hwvvJ5Fo6Pu=yKpRG5aUJ_-6UXJ_xcVPpQxSYVEA@mail.gmail.com>

Hi all,

Here are some of the major updates done to the new website in recent
months.

- Download links automatically updated from bitbucket/pypy download page.
- Supported libs automatically updated from bitbucket/pypy wiki.
- Fixed usability/linking issues with the sub section navigation.
- Code highlighting
- Write content in either ReST/raw text/HTML.
- Working twitter feed.
- Better performance overall.
- Migrated from ep.io to dotCloud.

www: http://pypy-nikuda.dotcloud.com/
hg: https://bitbucket.org/pypy/pypy-site
wiki: https://bitbucket.org/nikuda/pypy-site/wiki/Home

To address some of the issues brought up last time the new website was
discussed: One of the reasons for updating the design of the website was to
make it more accessible to end-users, while having the deeper technical
details available for those that need it. This is the reason behind
implementing the 'hidden' until wanted, or dig down approach in the
structure of the website. I understand this can seem unnecessary to people
who are very used to the project and how it works. Even though I don't see
a problem with it, it's only one possible solution and we can change things
around if we agree that this particular implementation is not suited.

The website is not finished and there are a few things that I want to
change with it, but I do believe there's a good working base there that can
be expanded on and improved. Approach is definitively an iterative one, so
feel free add suggestions here or in the wiki.

Thanks,
Ante
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120602/de190ae7/attachment.html>

From alex.gaynor at gmail.com  Sat Jun  2 04:40:31 2012
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 1 Jun 2012 21:40:31 -0500
Subject: [pypy-dev] New pypy website update
In-Reply-To: <CAPPs_vCyd1hwvvJ5Fo6Pu=yKpRG5aUJ_-6UXJ_xcVPpQxSYVEA@mail.gmail.com>
References: <CAPPs_vCyd1hwvvJ5Fo6Pu=yKpRG5aUJ_-6UXJ_xcVPpQxSYVEA@mail.gmail.com>
Message-ID: <CAFRnB2X76seyuALnri68pdZVEaEswzP1_LSadZ_XC4RXqbNEWg@mail.gmail.com>

On Fri, Jun 1, 2012 at 9:24 PM, ante <nikuda at gmail.com> wrote:

> Hi all,
>
> Here are some of the major updates done to the new website in recent
> months.
>
> - Download links automatically updated from bitbucket/pypy download page.
> - Supported libs automatically updated from bitbucket/pypy wiki.
> - Fixed usability/linking issues with the sub section navigation.
> - Code highlighting
> - Write content in either ReST/raw text/HTML.
> - Working twitter feed.
> - Better performance overall.
> - Migrated from ep.io to dotCloud.
>
> www: http://pypy-nikuda.dotcloud.com/
> hg: https://bitbucket.org/pypy/pypy-site
> wiki: https://bitbucket.org/nikuda/pypy-site/wiki/Home
>
> To address some of the issues brought up last time the new website was
> discussed: One of the reasons for updating the design of the website was to
> make it more accessible to end-users, while having the deeper technical
> details available for those that need it. This is the reason behind
> implementing the 'hidden' until wanted, or dig down approach in the
> structure of the website. I understand this can seem unnecessary to people
> who are very used to the project and how it works. Even though I don't see
> a problem with it, it's only one possible solution and we can change things
> around if we agree that this particular implementation is not suited.
>
> The website is not finished and there are a few things that I want to
> change with it, but I do believe there's a good working base there that can
> be expanded on and improved. Approach is definitively an iterative one, so
> feel free add suggestions here or in the wiki.
>
> Thanks,
> Ante
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
Hey, I haven't had a chance to look everything over, but I wanted to thank
you for all your hard work on this!

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120601/037dd8e2/attachment-0001.html>

From fijall at gmail.com  Sat Jun  2 11:40:10 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 2 Jun 2012 11:40:10 +0200
Subject: [pypy-dev] PyPy build bots?
In-Reply-To: <CAL36E+vkoEgUEzZKVpHXok1dONzyzRugfmxJ7dLZT1eAFrB_UA@mail.gmail.com>
References: <CAL36E+vkoEgUEzZKVpHXok1dONzyzRugfmxJ7dLZT1eAFrB_UA@mail.gmail.com>
Message-ID: <CAK5idxSERZRZ6x+92mHicxW7TDqzi0=Ri58zngWeTjfUpaH4Uw@mail.gmail.com>

On Fri, Jun 1, 2012 at 11:15 PM, Timothy Baldridge <tbaldridge at gmail.com>wrote:

> I'm at the point where I'd like to start somewhat regular builds of my
> pypy branch complete with JIT optimization. I tried downloading the
> build bot but ran into some errors. Does pypy have a build farm that
> branches can use? Any suggestions on the path-of-least resistance to
> fire off nightly builds?
>
> Thanks,
>
> Timothy
>

I don't think we have excessive build machines. Buildbot should just work
though. Have you consider a solution like travis-ci?

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120602/10a71e8a/attachment.html>

From anto.cuni at gmail.com  Sat Jun  2 11:54:14 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 02 Jun 2012 11:54:14 +0200
Subject: [pypy-dev] [pypy-commit] pypy win_ffi: fix test for jit
In-Reply-To: <20120601132533.E9C6D1C0274@cobra.cs.uni-duesseldorf.de>
References: <20120601132533.E9C6D1C0274@cobra.cs.uni-duesseldorf.de>
Message-ID: <4FC9E2C6.2030107@gmail.com>

Hello Matti,

On 06/01/2012 03:25 PM, mattip wrote:

> @@ -551,6 +552,9 @@
>              except ValueError, e:
>                  assert e.message == 'Procedure called with not enough ' + \
>                       'arguments (8 bytes missing) or wrong calling convention'
> +            except LLException:
> +                #jitted code raises this
> +                pass
>              else:
>                  assert 0, 'wrong calling convention should have raised'

I think you should at least check that the LLException you get is of the right
type, to avoid hiding an unrelated one you might get by chance.

ciao,
Anto

From fijall at gmail.com  Sat Jun  2 13:09:03 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 2 Jun 2012 13:09:03 +0200
Subject: [pypy-dev] New pypy website update
In-Reply-To: <CAFRnB2X76seyuALnri68pdZVEaEswzP1_LSadZ_XC4RXqbNEWg@mail.gmail.com>
References: <CAPPs_vCyd1hwvvJ5Fo6Pu=yKpRG5aUJ_-6UXJ_xcVPpQxSYVEA@mail.gmail.com>
	<CAFRnB2X76seyuALnri68pdZVEaEswzP1_LSadZ_XC4RXqbNEWg@mail.gmail.com>
Message-ID: <CAK5idxROG77YFt+Pt_SJrfD_fCpZ-_J8aT6jDDcQDvRXbPiyEg@mail.gmail.com>

On Sat, Jun 2, 2012 at 4:40 AM, Alex Gaynor <alex.gaynor at gmail.com> wrote:

>
>
> On Fri, Jun 1, 2012 at 9:24 PM, ante <nikuda at gmail.com> wrote:
>
>>  Hi all,
>>
>> Here are some of the major updates done to the new website in recent
>> months.
>>
>> - Download links automatically updated from bitbucket/pypy download page.
>>  - Supported libs automatically updated from bitbucket/pypy wiki.
>> - Fixed usability/linking issues with the sub section navigation.
>> - Code highlighting
>> - Write content in either ReST/raw text/HTML.
>> - Working twitter feed.
>> - Better performance overall.
>> - Migrated from ep.io to dotCloud.
>>
>> www: http://pypy-nikuda.dotcloud.com/
>> hg: https://bitbucket.org/pypy/pypy-site
>> wiki: https://bitbucket.org/nikuda/pypy-site/wiki/Home
>>
>> To address some of the issues brought up last time the new website was
>> discussed: One of the reasons for updating the design of the website was to
>> make it more accessible to end-users, while having the deeper technical
>> details available for those that need it. This is the reason behind
>> implementing the 'hidden' until wanted, or dig down approach in the
>> structure of the website. I understand this can seem unnecessary to people
>> who are very used to the project and how it works. Even though I don't see
>> a problem with it, it's only one possible solution and we can change things
>> around if we agree that this particular implementation is not suited.
>>
>> The website is not finished and there are a few things that I want to
>> change with it, but I do believe there's a good working base there that can
>> be expanded on and improved. Approach is definitively an iterative one, so
>> feel free add suggestions here or in the wiki.
>>
>> Thanks,
>> Ante
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>>
> Hey, I haven't had a chance to look everything over, but I wanted to thank
> you for all your hard work on this!
>
> Alex
>
> Same here. I'm sorry we haven't been responsive enough (in fact your mails
are the oldest starred mails in my inbox).

Thanks for all the hard work, let's see what we can do!

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120602/7d30868e/attachment.html>

From alex.pyattaev at gmail.com  Sun Jun  3 10:16:05 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Sun, 03 Jun 2012 11:16:05 +0300
Subject: [pypy-dev] Python FFI
In-Reply-To: <2392225.RKfMf6lBSA@robert-zaremba-bc>
References: <mailman.17410.1337094715.3036.pypy-dev@python.org>
	<CAMSv6X2TmhyOEFz7ukbvEpH0KpU=gCvXUFp=cnUDMsBqte-6Sw@mail.gmail.com>
	<2392225.RKfMf6lBSA@robert-zaremba-bc>
Message-ID: <148749038.6f1WlqRMH2@hunter-laptop>

So, when would users be able to test it? I'd like to compare the overhead with 
ctypes and cppyy for instance. I also wonder if I'd be able to send Python 
objects to the c++ domain for storage in a container (e.g. a vector).
BR,
Alex
keskiviikko 30 toukokuu 2012 18:00:16 Robert Zaremba kirjoitti:
> Thanks for clarification.  I've been wondering why to dobule the  work.
> 
> I'm also really fascinated about this idea, and that i don't need to use
> separate tool.
> It looks really great! Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120603/947c147c/attachment.html>

From arigo at tunes.org  Sun Jun  3 11:57:24 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 3 Jun 2012 11:57:24 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
Message-ID: <CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>

Hi Uwe,

On Tue, May 1, 2012 at 1:05 AM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> I edited / hacked the sources and managed to compile PyPy under Cygwin based
> on Python 2.6.7 and gcc-4.5.3. If you find this of interest, the patches are
> available at http://www.tux.org/~mayer/cygwin/pypy/.

Thanks!  I finally took the time to review and merge the patch.

I didn't include any change to lib-python: they are changes that look
like they should also be done to CPython..?  I may be mistaken, but it
looks like fixes for tests that would also fail on CPython+cygwin.  If
that's the case, then it should be reported and fixed in CPython.


A bient?t,

Armin.

From alex.pyattaev at gmail.com  Sun Jun  3 01:33:15 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Sun, 03 Jun 2012 02:33:15 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206011548200.1119@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<1543846.T9Sm08LvAA@hunter-laptop>
	<alpine.LNX.2.00.1206011548200.1119@pc3-wlav.lbl.gov>
Message-ID: <10542917.ncyhjCfde6@hunter-laptop>

> good you found a workaround, as using cpyext from cppyy is turning out to be
> a bigger can of worms that I expected. :} Will take a tad bit longer: I
> have some working code, but only post-translation. For some reason, the
> resulting objects look nothing as expected in the test and as passed, they
> go into the CPython interpreter (this is a test, after all), which then
> bombs.
Cpyext is not needed with cppyy as far as I understand, is it?
> Post-translation, it's looking fine, though, and working as expected.
> 
> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
Performance was originally the point, right? Or am I missing something (lol)?

> My guess is that in the micro-benchmark, the JIT gets no chance of making a
> trace out of the loop iteration, since the loop iteration requires the
> PyObject in every call (the PyObject that I'm passing is the iterator
> variable itself). I'll have to look into more detail ....
I am giving you my benchmark code, right now it uses dictionary to map 
pyobject id into objects when they leave c++ domain. Obviously, you add about 
O(2*log(N)) overhead to the code, but that should not be too much anyway. If 
that can be done in rpython, it could probably be made to work massively 
faster. Like a module that does object caching for the storage of  objects 
that would be sent to c++ domain. Then in c++ one can do as simple as have an 
long int index that can be used to resolve back to byobject when the job is 
done.
It also does classical SWIG typemap to do the same job. to use swig use 
SConstruct build. Or just use the run.sh script to get all the stuff 
immediately.

> > With respect to genreflex command, you could probably warn people that
> > they
> > should use gcc 4.3, not 4.6, cuz gccxml, used by genreflex, is not
> > compatible with newer versions of STL, and crashes on standard headers.
> 
> That shouldn't be ... on my box I have gcc 4.6.2 and that does not pose any
> problems. What version of gccxml do you have? Mine's at 0.9.0.

mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have older 
gcc in a different slot? 

> > Also, any headers
> > that are not absolutely necessary to define the interface to Python should
> > be removed if possible, that does make life a lot easier, even with SWIG,
> > because it makes wrappers smaller.
> 
> Have you looked into providing a selection.xml to genreflex to select the
> needed classes? Also, another common option is to collect all desired
> headers in a single header file (with simple #include's) and feed that to
> genreflex, together with a selection.xml that simply uses "*" patterns.
The point is that if the header is included, and it confuses the parser, then 
the patterns have no effect. So it is better not to have extra headers like Qt 
library=)

Alex.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120603/f397c267/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: reflex.tar.gz
Type: application/x-compressed-tar
Size: 119967 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120603/f397c267/attachment-0001.bin>

From arigo at tunes.org  Sun Jun  3 15:46:49 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 3 Jun 2012 15:46:49 +0200
Subject: [pypy-dev] [Repost] PyPy 1.8 50% slower than PyPy 1.7 on my
	test case
In-Reply-To: <CAMSv6X05aNCq47okQpucFHSsburL3gPeXr4JXNb29hxn3YAtVg@mail.gmail.com>
References: <CABqV1b5WkZ8m9iifx4Prm0eKUV8df5+RLkiUXtMJG9dtMKODew@mail.gmail.com>
	<CAMSv6X05aNCq47okQpucFHSsburL3gPeXr4JXNb29hxn3YAtVg@mail.gmail.com>
Message-ID: <CAMSv6X3zy=ieggBVfSZM=R5UAGHYhyzkfLY7x1gQ-UMPL3hVEw@mail.gmail.com>

Hi Laurent,

Follow-up on the issue with your observed slow-down,
https://bugs.pypy.org/issue1051 :

It seems to be faster again on a pypy trunk nowadays than it was on
pypy 1.8; actually it is faster now than with pypy 1.7.  Can you
report if you're experiencing the same results on Windows?

Do you mind if we steal hexiom2.py and level36.txt and put them in
https://bitbucket.org/pypy/benchmarks, so that the program will be run
nightly and we can spot speed regressions directly?

Thanks!


Armin.

From arigo at tunes.org  Sun Jun  3 16:28:52 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 3 Jun 2012 16:28:52 +0200
Subject: [pypy-dev] Python FFI
In-Reply-To: <148749038.6f1WlqRMH2@hunter-laptop>
References: <mailman.17410.1337094715.3036.pypy-dev@python.org>
	<CAMSv6X2TmhyOEFz7ukbvEpH0KpU=gCvXUFp=cnUDMsBqte-6Sw@mail.gmail.com>
	<2392225.RKfMf6lBSA@robert-zaremba-bc>
	<148749038.6f1WlqRMH2@hunter-laptop>
Message-ID: <CAMSv6X10q+1c5c_RrJUo1uBp9EVqGPRLBQPQKXsUeYkOxti0eQ@mail.gmail.com>

Hi Alex,

On Sun, Jun 3, 2012 at 10:16 AM, Alex Pyattaev <alex.pyattaev at gmail.com> wrote:
> So, when would users be able to test it? I'd like to compare the overhead
> with ctypes and cppyy for instance.

The first version is ready for testing at https://github.com/arigo/ffi
.  Note that the goal is not to interface with C++ but only with C.
In this pre-0.1-release, it supports the same things as a basic
ctypes, minus all the Windows calling convention stuff.  We're going
to work now on non-ctypes issues like structs without a fixed known
layout, macros, etc. (listed in the README.md file).

Reading just the file names in 'ctypes/test/test_*.py', I should
stress again that the scale of the project is completely different
than ctypes: it's meant as a minimal interface to C offering a C-like
feeling, with absolutely no extra features on top.

See the 'demo/readdir.py' file for a quick introduction.  If you
already know independently Python, C, and (for this example) did at
some point read "man openat" on Linux, then this code should be
self-explanatory and kind of obvious.


A bient?t,

Armin.

From alex.pyattaev at gmail.com  Mon Jun  4 00:38:03 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Mon, 04 Jun 2012 01:38:03 +0300
Subject: [pypy-dev] Python FFI
In-Reply-To: <CAMSv6X10q+1c5c_RrJUo1uBp9EVqGPRLBQPQKXsUeYkOxti0eQ@mail.gmail.com>
References: <mailman.17410.1337094715.3036.pypy-dev@python.org>
	<148749038.6f1WlqRMH2@hunter-laptop>
	<CAMSv6X10q+1c5c_RrJUo1uBp9EVqGPRLBQPQKXsUeYkOxti0eQ@mail.gmail.com>
Message-ID: <2983487.rZ4ePPHiON@hunter-laptop>

Okay,
 i see the issue now. Basically i was under an impression that FFI was somehow 
a more elaborate thing... Sorry for the bother, the actual thing i need is 
more like cppyy, but working =) Anyways, might be actually interesting to see 
it work with SWIG, since it uses C-like functions to wrap C++ classes. But, as 
you have said, it is pre-alpha, so no SWIG as of now...
BR,
Alex.
sunnuntai 03 kes?kuu 2012 16:28:52 Armin Rigo kirjoitti:
> Hi Alex,
> 
> On Sun, Jun 3, 2012 at 10:16 AM, Alex Pyattaev <alex.pyattaev at gmail.com> 
wrote:
> > So, when would users be able to test it? I'd like to compare the overhead
> > with ctypes and cppyy for instance.
> 
> The first version is ready for testing at https://github.com/arigo/ffi
> .  Note that the goal is not to interface with C++ but only with C.
> In this pre-0.1-release, it supports the same things as a basic
> ctypes, minus all the Windows calling convention stuff.  We're going
> to work now on non-ctypes issues like structs without a fixed known
> layout, macros, etc. (listed in the README.md file).
> 
> Reading just the file names in 'ctypes/test/test_*.py', I should
> stress again that the scale of the project is completely different
> than ctypes: it's meant as a minimal interface to C offering a C-like
> feeling, with absolutely no extra features on top.
> 
> See the 'demo/readdir.py' file for a quick introduction.  If you
> already know independently Python, C, and (for this example) did at
> some point read "man openat" on Linux, then this code should be
> self-explanatory and kind of obvious.
> 
> 
> A bient?t,
> 
> Armin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/f79e55f7/attachment.html>

From alex.pyattaev at gmail.com  Mon Jun  4 01:40:35 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Mon, 04 Jun 2012 02:40:35 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <10542917.ncyhjCfde6@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1206011548200.1119@pc3-wlav.lbl.gov>
	<10542917.ncyhjCfde6@hunter-laptop>
Message-ID: <2893536.M4AdCYZRgx@hunter-laptop>

Please, mention in the manual, for the mortals, that the reflection library 
has to be linked with -lReflex, so that they do not have to figure that out on 
their own=)
BR,
Alex.
sunnuntai 03 kes?kuu 2012 02:33:15 Alex Pyattaev kirjoitti:

> good you found a workaround, as using cpyext from cppyy is turning out to be
> a bigger can of worms that I expected. :} Will take a tad bit longer: I
> have some working code, but only post-translation. For some reason, the
> resulting objects look nothing as expected in the test and as passed, they
> go into the CPython interpreter (this is a test, after all), which then
> bombs.
Cpyext is not needed with cppyy as far as I understand, is it?
> Post-translation, it's looking fine, though, and working as expected.
> 
> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
Performance was originally the point, right? Or am I missing something (lol)?

> My guess is that in the micro-benchmark, the JIT gets no chance of making a
> trace out of the loop iteration, since the loop iteration requires the
> PyObject in every call (the PyObject that I'm passing is the iterator
> variable itself). I'll have to look into more detail ....
I am giving you my benchmark code, right now it uses dictionary to map 
pyobject id into objects when they leave c++ domain. Obviously, you add about 
O(2*log(N)) overhead to the code, but that should not be too much anyway. If 
that can be done in rpython, it could probably be made to work massively 
faster. Like a module that does object caching for the storage of  objects 
that would be sent to c++ domain. Then in c++ one can do as simple as have an 
long int index that can be used to resolve back to byobject when the job is 
done.
It also does classical SWIG typemap to do the same job. to use swig use 
SConstruct build. Or just use the run.sh script to get all the stuff 
immediately.

> > With respect to genreflex command, you could probably warn people that
> > they
> > should use gcc 4.3, not 4.6, cuz gccxml, used by genreflex, is not
> > compatible with newer versions of STL, and crashes on standard headers.
> 
> That shouldn't be ... on my box I have gcc 4.6.2 and that does not pose any
> problems. What version of gccxml do you have? Mine's at 0.9.0.

mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have older 
gcc in a different slot? 

> > Also, any headers
> > that are not absolutely necessary to define the interface to Python should
> > be removed if possible, that does make life a lot easier, even with SWIG,
> > because it makes wrappers smaller.
> 
> Have you looked into providing a selection.xml to genreflex to select the
> needed classes? Also, another common option is to collect all desired
> headers in a single header file (with simple #include's) and feed that to
> genreflex, together with a selection.xml that simply uses "*" patterns.
The point is that if the header is included, and it confuses the parser, then 
the patterns have no effect. So it is better not to have extra headers like Qt 
library=)

Alex.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/101c4070/attachment-0001.html>

From wlavrijsen at lbl.gov  Mon Jun  4 01:44:19 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Sun, 3 Jun 2012 16:44:19 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <10542917.ncyhjCfde6@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<1543846.T9Sm08LvAA@hunter-laptop>
	<alpine.LNX.2.00.1206011548200.1119@pc3-wlav.lbl.gov>
	<10542917.ncyhjCfde6@hunter-laptop>
Message-ID: <alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>

Hi Alex,

>> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
> Performance was originally the point, right? Or am I missing something (lol)?

well, remember that there are two problems here: 1) since there are no
PyObject's as such in pypy-c (they have different internal structure), one
has the create them, to have a PyObject* pointer to them; and keep track of
them to mix the reference counting scheme with the garbage collection scheme
(this is all done in/by cpyext). And 2) now that the object is created,
pinning down a variable in the .py code, the JIT can't unbox it.

The example I used, which was like so:

       for i in range(N):
           a.add_data(i)

and is rather pathological, of course. But then, all micro-benches are.

Conversely, doing:

       l = []
       for i in range(N):
           l.append(i)
       a.add_data(l)

is very speedy, presumably (I don't know, guessing here as it may also be
that there is another optimization for the list that I can't prevent even
with the append() calls) because now the cpyext code is in a loop in and
of itself, so e.g. lookup of the type of the objects in the container (all
the same here) can become constant.

> I am giving you my benchmark code

Which clearly is also pathological. :) I can't manage to run the SWIG test
from pypy, though? I tried by adding cpyext, but no dice?

Anyway, you'll be happy to know that:

CPPYY
Created container
No more items

real	0m2.236s
user	0m2.156s
sys	0m0.071s

Cpython
Created container
No more items

real	0m2.758s
user	0m2.574s
sys	0m0.174s

(I verified with your debug=1 that the objects appear.)

But that improvement is (I'm sure) only b/c all the other stuff you do
around it with manipulating the dict on the python side. That does get
JIT'ed just fine.

Btw., I had to jack up N by 10x, otherwise all numbers were just noise. :}

> right now it uses dictionary to map
> pyobject id into objects when they leave c++ domain. Obviously, you add about
> O(2*log(N)) overhead to the code, but that should not be too much anyway. If
> that can be done in rpython, it could probably be made to work massively
> faster. Like a module that does object caching for the storage of  objects
> that would be sent to c++ domain. Then in c++ one can do as simple as have an
> long int index that can be used to resolve back to byobject when the job is
> done.

>From a quick read of the cpyext code, that is rather similar to what is there.

But as per above, the problem isn't with the extra overhead, the problem is
with the JIT being prevented to do any work (and with pypy-c being slower
than CPython in pure interpreted mode).

> mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have
> older gcc in a different slot?

I do, but those should not be accessible from my pypy work environment. I
did rebuild it. Not sure whether that matters. I'll update the docs showing
how a different set of headers from another compiler can be selected.

> The point is that if the header is included, and it confuses the parser, then
> the patterns have no effect. So it is better not to have extra headers like Qt
> library=)

Qt is a story in and of itself, of course. :) Actually, what folks here do,
is to link any code from external C++ through python to Qt (C++) by making
use of the fact that both ends can handle opaque pointers, represented as
long values. (The combination of addressof and bind_object in cppyy.)

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From alex.pyattaev at gmail.com  Mon Jun  4 02:02:49 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Mon, 04 Jun 2012 03:02:49 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
Message-ID: <2785332.RJWjHmEeLe@hunter-laptop>

Okay, with respect to the SWIG + pypy, works for me, and about 4x slower then 
cppyy or cpython ( which show same results more or less).

time cppyy ./cppyy_timings.py 

real    0m0.265s
user    0m0.225s
sys     0m0.040s


time pypy ./pypy_timings.py 
real    0m2.424s
user    0m2.338s
sys     0m0.086s

time python2 ./pypy_timings.py 
real    0m0.414s
user    0m0.390s
sys     0m0.024s

So apparently pypy is dramatically bad with SWIG. cppyy seems to work nice. I 
have, however, encountered an interesting problem. Say I have a dictionary 
built. I'd like to know which symbols/classes are actually available (if any), 
but I can not do that. The implication is that in case your library does not 
have any symbols in it after SUCCESSFUL build, there is no way to figure out 
why and how that happened. Please advise. Also I can bench the entire thing on 
a real, quite large app that makes intensive use of pypy and also runs on 
python2 if needed, but for that we would need to figure why no classes are 
visible in the dictionary=)

BR,
Alex

sunnuntai 03 kes?kuu 2012 16:44:19 wlavrijsen at lbl.gov kirjoitti:
> Hi Alex,
> 
> >> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
> > 
> > Performance was originally the point, right? Or am I missing something
> > (lol)?
> well, remember that there are two problems here: 1) since there are no
> PyObject's as such in pypy-c (they have different internal structure), one
> has the create them, to have a PyObject* pointer to them; and keep track of
> them to mix the reference counting scheme with the garbage collection scheme
> (this is all done in/by cpyext). And 2) now that the object is created,
> pinning down a variable in the .py code, the JIT can't unbox it.
> 
> The example I used, which was like so:
> 
>        for i in range(N):
>            a.add_data(i)
> 
> and is rather pathological, of course. But then, all micro-benches are.
> 
> Conversely, doing:
> 
>        l = []
>        for i in range(N):
>            l.append(i)
>        a.add_data(l)
> 
> is very speedy, presumably (I don't know, guessing here as it may also be
> that there is another optimization for the list that I can't prevent even
> with the append() calls) because now the cpyext code is in a loop in and
> of itself, so e.g. lookup of the type of the objects in the container (all
> the same here) can become constant.
> 
> > I am giving you my benchmark code
> 
> Which clearly is also pathological. :) I can't manage to run the SWIG test
> from pypy, though? I tried by adding cpyext, but no dice?
> 
> Anyway, you'll be happy to know that:
> 
> CPPYY
> Created container
> No more items
> 
> real	0m2.236s
> user	0m2.156s
> sys	0m0.071s
> 
> Cpython
> Created container
> No more items
> 
> real	0m2.758s
> user	0m2.574s
> sys	0m0.174s
> 
> (I verified with your debug=1 that the objects appear.)
> 
> But that improvement is (I'm sure) only b/c all the other stuff you do
> around it with manipulating the dict on the python side. That does get
> JIT'ed just fine.
> 
> Btw., I had to jack up N by 10x, otherwise all numbers were just noise. :}
> 
> > right now it uses dictionary to map
> > pyobject id into objects when they leave c++ domain. Obviously, you add
> > about O(2*log(N)) overhead to the code, but that should not be too much
> > anyway. If that can be done in rpython, it could probably be made to work
> > massively faster. Like a module that does object caching for the storage
> > of  objects that would be sent to c++ domain. Then in c++ one can do as
> > simple as have an long int index that can be used to resolve back to
> > byobject when the job is done.
> 
> From a quick read of the cpyext code, that is rather similar to what is
> there.
> 
> But as per above, the problem isn't with the extra overhead, the problem is
> with the JIT being prevented to do any work (and with pypy-c being slower
> than CPython in pure interpreted mode).
> 
> > mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have
> > older gcc in a different slot?
> 
> I do, but those should not be accessible from my pypy work environment. I
> did rebuild it. Not sure whether that matters. I'll update the docs showing
> how a different set of headers from another compiler can be selected.
> 
> > The point is that if the header is included, and it confuses the parser,
> > then the patterns have no effect. So it is better not to have extra
> > headers like Qt library=)
> 
> Qt is a story in and of itself, of course. :) Actually, what folks here do,
> is to link any code from external C++ through python to Qt (C++) by making
> use of the fact that both ends can handle opaque pointers, represented as
> long values. (The combination of addressof and bind_object in cppyy.)
> 
> Best regards,
>             Wim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/fe940ba1/attachment-0001.html>

From alex.pyattaev at gmail.com  Mon Jun  4 02:29:45 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Mon, 04 Jun 2012 03:29:45 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
Message-ID: <2218537.2NmGT9yelu@hunter-laptop>

I think I've figured it with the missing classes. When supplied with multiple 
headers, genreflex apparently only saves the last one of them...Obviously, 
that can be worked around, but still seems like a bug or just me not finding 
some manual.
BR,
Alex.
sunnuntai 03 kes?kuu 2012 16:44:19 wlavrijsen at lbl.gov kirjoitti:
> Hi Alex,
> 
> >> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
> > 
> > Performance was originally the point, right? Or am I missing something
> > (lol)?
> well, remember that there are two problems here: 1) since there are no
> PyObject's as such in pypy-c (they have different internal structure), one
> has the create them, to have a PyObject* pointer to them; and keep track of
> them to mix the reference counting scheme with the garbage collection scheme
> (this is all done in/by cpyext). And 2) now that the object is created,
> pinning down a variable in the .py code, the JIT can't unbox it.
> 
> The example I used, which was like so:
> 
>        for i in range(N):
>            a.add_data(i)
> 
> and is rather pathological, of course. But then, all micro-benches are.
> 
> Conversely, doing:
> 
>        l = []
>        for i in range(N):
>            l.append(i)
>        a.add_data(l)
> 
> is very speedy, presumably (I don't know, guessing here as it may also be
> that there is another optimization for the list that I can't prevent even
> with the append() calls) because now the cpyext code is in a loop in and
> of itself, so e.g. lookup of the type of the objects in the container (all
> the same here) can become constant.
> 
> > I am giving you my benchmark code
> 
> Which clearly is also pathological. :) I can't manage to run the SWIG test
> from pypy, though? I tried by adding cpyext, but no dice?
> 
> Anyway, you'll be happy to know that:
> 
> CPPYY
> Created container
> No more items
> 
> real	0m2.236s
> user	0m2.156s
> sys	0m0.071s
> 
> Cpython
> Created container
> No more items
> 
> real	0m2.758s
> user	0m2.574s
> sys	0m0.174s
> 
> (I verified with your debug=1 that the objects appear.)
> 
> But that improvement is (I'm sure) only b/c all the other stuff you do
> around it with manipulating the dict on the python side. That does get
> JIT'ed just fine.
> 
> Btw., I had to jack up N by 10x, otherwise all numbers were just noise. :}
> 
> > right now it uses dictionary to map
> > pyobject id into objects when they leave c++ domain. Obviously, you add
> > about O(2*log(N)) overhead to the code, but that should not be too much
> > anyway. If that can be done in rpython, it could probably be made to work
> > massively faster. Like a module that does object caching for the storage
> > of  objects that would be sent to c++ domain. Then in c++ one can do as
> > simple as have an long int index that can be used to resolve back to
> > byobject when the job is done.
> 
> From a quick read of the cpyext code, that is rather similar to what is
> there.
> 
> But as per above, the problem isn't with the extra overhead, the problem is
> with the JIT being prevented to do any work (and with pypy-c being slower
> than CPython in pure interpreted mode).
> 
> > mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have
> > older gcc in a different slot?
> 
> I do, but those should not be accessible from my pypy work environment. I
> did rebuild it. Not sure whether that matters. I'll update the docs showing
> how a different set of headers from another compiler can be selected.
> 
> > The point is that if the header is included, and it confuses the parser,
> > then the patterns have no effect. So it is better not to have extra
> > headers like Qt library=)
> 
> Qt is a story in and of itself, of course. :) Actually, what folks here do,
> is to link any code from external C++ through python to Qt (C++) by making
> use of the fact that both ends can handle opaque pointers, represented as
> long values. (The combination of addressof and bind_object in cppyy.)
> 
> Best regards,
>             Wim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/ce5f5e81/attachment.html>

From laurentvaucher at gmail.com  Mon Jun  4 07:02:53 2012
From: laurentvaucher at gmail.com (Laurent Vaucher)
Date: Mon, 4 Jun 2012 07:02:53 +0200
Subject: [pypy-dev] [Repost] PyPy 1.8 50% slower than PyPy 1.7 on my
	test case
In-Reply-To: <CAMSv6X3zy=ieggBVfSZM=R5UAGHYhyzkfLY7x1gQ-UMPL3hVEw@mail.gmail.com>
References: <CABqV1b5WkZ8m9iifx4Prm0eKUV8df5+RLkiUXtMJG9dtMKODew@mail.gmail.com>
	<CAMSv6X05aNCq47okQpucFHSsburL3gPeXr4JXNb29hxn3YAtVg@mail.gmail.com>
	<CAMSv6X3zy=ieggBVfSZM=R5UAGHYhyzkfLY7x1gQ-UMPL3hVEw@mail.gmail.com>
Message-ID: <CABqV1b7RNSkcHj_eeDyTd0OtgHvAr4k46WedOr2vhfViUpn6pA@mail.gmail.com>

That's great news! Thanks!

I don't mind at all your using my test files. "I hereby grant you all the
rights you need to do that, as I am the sole author of this code." I'm
coding for fun, not to assert any copyright, patent or anything.

I'll try the latest trunk version and tell you how it goes.

Thanks again.
Laurent.

On Sun, Jun 3, 2012 at 3:46 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Laurent,
>
> Follow-up on the issue with your observed slow-down,
> https://bugs.pypy.org/issue1051 :
>
> It seems to be faster again on a pypy trunk nowadays than it was on
> pypy 1.8; actually it is faster now than with pypy 1.7.  Can you
> report if you're experiencing the same results on Windows?
>
> Do you mind if we steal hexiom2.py and level36.txt and put them in
> https://bitbucket.org/pypy/benchmarks, so that the program will be run
> nightly and we can spot speed regressions directly?
>
> Thanks!
>
>
> Armin.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/2db784f9/attachment-0001.html>

From laurentvaucher at gmail.com  Mon Jun  4 15:29:15 2012
From: laurentvaucher at gmail.com (Laurent Vaucher)
Date: Mon, 4 Jun 2012 15:29:15 +0200
Subject: [pypy-dev] [Repost] PyPy 1.8 50% slower than PyPy 1.7 on my
	test case
In-Reply-To: <CABqV1b7RNSkcHj_eeDyTd0OtgHvAr4k46WedOr2vhfViUpn6pA@mail.gmail.com>
References: <CABqV1b5WkZ8m9iifx4Prm0eKUV8df5+RLkiUXtMJG9dtMKODew@mail.gmail.com>
	<CAMSv6X05aNCq47okQpucFHSsburL3gPeXr4JXNb29hxn3YAtVg@mail.gmail.com>
	<CAMSv6X3zy=ieggBVfSZM=R5UAGHYhyzkfLY7x1gQ-UMPL3hVEw@mail.gmail.com>
	<CABqV1b7RNSkcHj_eeDyTd0OtgHvAr4k46WedOr2vhfViUpn6pA@mail.gmail.com>
Message-ID: <CABqV1b5Wao5FfepOKf4wkg=Xz5rA_YV=m96ZwQdDgjHP0wuCNg@mail.gmail.com>

Well, with the latest Windows nightly bundle, it's now 10-20% faster than
1.7 and almost twice as fast as 1.8, depending on the duration of the test.
I don't know what you did, but you did it damn well!

Bravo.
Laurent.

On Mon, Jun 4, 2012 at 7:02 AM, Laurent Vaucher <laurentvaucher at gmail.com>wrote:

> That's great news! Thanks!
>
> I don't mind at all your using my test files. "I hereby grant you all the
> rights you need to do that, as I am the sole author of this code." I'm
> coding for fun, not to assert any copyright, patent or anything.
>
> I'll try the latest trunk version and tell you how it goes.
>
> Thanks again.
> Laurent.
>
>
> On Sun, Jun 3, 2012 at 3:46 PM, Armin Rigo <arigo at tunes.org> wrote:
>
>> Hi Laurent,
>>
>> Follow-up on the issue with your observed slow-down,
>> https://bugs.pypy.org/issue1051 :
>>
>> It seems to be faster again on a pypy trunk nowadays than it was on
>> pypy 1.8; actually it is faster now than with pypy 1.7.  Can you
>> report if you're experiencing the same results on Windows?
>>
>> Do you mind if we steal hexiom2.py and level36.txt and put them in
>> https://bitbucket.org/pypy/benchmarks, so that the program will be run
>> nightly and we can spot speed regressions directly?
>>
>> Thanks!
>>
>>
>> Armin.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/8fea492f/attachment.html>

From arigo at tunes.org  Mon Jun  4 15:33:22 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Jun 2012 15:33:22 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
Message-ID: <CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>

Hi Martijn, hi Holger,

On 6/1/12, Martijn Faassen <faassen at startifact.com> wrote:
> Concerning performance overhead of proxies, lifecycle issues would be
> tricky

If, hypothetically speaking, there is someone interested in writing a
PyPy solution where both a Python2 and a Python3 interpreter are
running in the same process, then you gain the advantage of having
only one GC to run both.  At least it transparently solves the issues
of lifetime and reference cycles.  (You also have for free only one
JIT, which can do cross-language optimizations like inlining a Python2
function into a Python3 context or vice-versa).  I see these two
points as benefits that you don't have in any multi-process solution.

It would require some work on the PyPy side, and I'm not aware of
anybody ready to invest time in that, but it shouldn't be particularly
hard (once PyPy's Python3 interpreter is ready, and once people agree
about which API to use to do cross-language calls.)


A bient?t,

Armin.

From faassen at startifact.com  Mon Jun  4 15:35:22 2012
From: faassen at startifact.com (Martijn Faassen)
Date: Mon, 4 Jun 2012 15:35:22 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
Message-ID: <CAGT4ZFjiPSxs8oM4mODzy3wkXohs-ppZaQBMKgyris4e9csEOQ@mail.gmail.com>

Hey Armin,

Cool, having a shared GC and a shared JIT would be pretty neat features indeed!

Regards,

Martijn

From fuzzyman at gmail.com  Mon Jun  4 16:26:15 2012
From: fuzzyman at gmail.com (Michael Foord)
Date: Mon, 4 Jun 2012 15:26:15 +0100
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
Message-ID: <CAKCKLWwpeKBOtLVSFX=Owi=w67YDCTPX9sYjPue-E_BjEdgp8A@mail.gmail.com>

On 4 June 2012 14:33, Armin Rigo <arigo at tunes.org> wrote:

> Hi Martijn, hi Holger,
>
> On 6/1/12, Martijn Faassen <faassen at startifact.com> wrote:
> > Concerning performance overhead of proxies, lifecycle issues would be
> > tricky
>
> If, hypothetically speaking, there is someone interested in writing a
> PyPy solution where both a Python2 and a Python3 interpreter are
> running in the same process, then you gain the advantage of having
> only one GC to run both.  At least it transparently solves the issues
> of lifetime and reference cycles.  (You also have for free only one
> JIT, which can do cross-language optimizations like inlining a Python2
> function into a Python3 context or vice-versa).  I see these two
> points as benefits that you don't have in any multi-process solution.
>
> It would require some work on the PyPy side, and I'm not aware of
> anybody ready to invest time in that, but it shouldn't be particularly
> hard (once PyPy's Python3 interpreter is ready, and once people agree
> about which API to use to do cross-language calls.)
>
>
>

Having multiple interpreter instances within a single process allows for
lots of interesting possibilities. IronPython permits this and it was used
by Resolver One - just one of the reasons (along with the GIL) that it
would have been much harder to write Resolver One in CPython than in
IronPython (although .NET was chosen as a platform before Python was chosen
as an implementation language).

Allowing Python 2 and Python 3 to live within the same process would be
very interesting.

All the best,

Michael Foord


> A bient?t,
>
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>



-- 

http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/c6cf39f8/attachment.html>

From holger at merlinux.eu  Mon Jun  4 16:27:52 2012
From: holger at merlinux.eu (holger krekel)
Date: Mon, 4 Jun 2012 14:27:52 +0000
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
Message-ID: <20120604142752.GH11942@merlinux.eu>

Hi Armin, Martijn,

On Mon, Jun 04, 2012 at 15:33 +0200, Armin Rigo wrote:
> Hi Martijn, hi Holger,
> 
> On 6/1/12, Martijn Faassen <faassen at startifact.com> wrote:
> > Concerning performance overhead of proxies, lifecycle issues would be
> > tricky
> 
> If, hypothetically speaking, there is someone interested in writing a
> PyPy solution where both a Python2 and a Python3 interpreter are
> running in the same process, then you gain the advantage of having
> only one GC to run both.  At least it transparently solves the issues
> of lifetime and reference cycles.  (You also have for free only one
> JIT, which can do cross-language optimizations like inlining a Python2
> function into a Python3 context or vice-versa).  I see these two
> points as benefits that you don't have in any multi-process solution.
> 
> It would require some work on the PyPy side, and I'm not aware of
> anybody ready to invest time in that, but it shouldn't be particularly
> hard (once PyPy's Python3 interpreter is ready, and once people agree
> about which API to use to do cross-language calls.)

Good point, PyPy could indeed bring considerable performance and
resource benefits and avoid distributed GC issues.  Of course there also are
deployment and API questions that could maybe better first be tackled outside
of PyPy in the form of a prototype that ignores GC issues.

I am wondering, however, how many people really have that py2/py3 need 
and how much they care about performance in such situations. OTOH,
a good solution could trigger needs - that's how a lot of technology 
development works, anyway :)

best,
holger


> A bient?t,
> 
> Armin.
> 

From arigo at tunes.org  Mon Jun  4 16:59:16 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 4 Jun 2012 16:59:16 +0200
Subject: [pypy-dev] IOError with getpass() in django
In-Reply-To: <jq79bh$gkv$1@dough.gmane.org>
References: <4FC69456.7080408@gmail.com>
	<CAK5idxQrW-9-k8xOWWY44J3giqgt+z8O6tz6-7HBGHTDeP_TXg@mail.gmail.com>
	<jq79bh$gkv$1@dough.gmane.org>
Message-ID: <CAMSv6X0WHaN0B8SeGA=FqAyz-sjRp8LD3Cs8nkytMoQWyhUkkQ@mail.gmail.com>

Hi all,

I finally fixed the bug, by ignoring flush() entirely in case they
don't manage to do lseek().


A bient?t,

Armin.

On 5/31/12, Gelonida N <gelonida at gmail.com> wrote:
> On 05/31/2012 09:15 AM, Maciej Fijalkowski wrote:
>> On Wed, May 30, 2012 at 11:42 PM, Gelonida N <gelonida at gmail.com
>> <mailto:gelonida at gmail.com>> wrote:
>  >
>>     I wanted to make my very first test with pypy by using django and
>>     sqlite3 on a Linux host (Ubuntu 12.04)
>>
>>     when running ./manage.py syncdb I'm asked to enter a password and
>>     pypy aborts as soon as I press enter.
>>
>>           File
>>
>> "/home/gelonida/mypypy/site-__packages/django/contrib/auth/__management/commands/__createsuperuser.py",
>>         line 109, in handle
>>             password = getpass.getpass()
>>           File "/usr/lib/pypy/lib-python/2.7/__getpass.py", line 74, in
>>         unix_getpass
>>             stream.flush()  # issue7208
>>         IOError: [Errno 29] Illegal seek: '<fdopen>'
>>
>>     Version info:
>>     Python 2.7.2 (1.8+dfsg-2, Feb 19 2012, 19:18:08)
>>     [PyPy 1.8.0 with GCC 4.6.2]
>>
>>     Django version: 1.3.1
>>
>>
>>     I even found a bug, which seems related (
>>     https://bugs.pypy.org/issue872 )
>>
>>     However I don't understand the impact and what this means exactly.
>>     Should I try to install another pypy (and if yes, how can I install
>>     it next to an existing one on Ubuntu 12.04)?
>>
>>     Thanks in advance for your help.
>>
>>
>> No, it's a bug and we did not fix it yet. I'll have a look some time
>> today.
>
> Thanks for your answer.
> Shouldn't this affect all users trying to use Django on a new project?
> I didn't find any way by means of command line arguments to avoid the
> call to getpass()
>
> How do others work around it?
>
> What I did in the end was, that  I just called the first
> "./manage.py syncdb" with ordinary CPython.
>
> Subsequent calls won't call getpass() anymore and should thus be fine.
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

From faassen at startifact.com  Mon Jun  4 18:31:01 2012
From: faassen at startifact.com (Martijn Faassen)
Date: Mon, 4 Jun 2012 18:31:01 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <20120604142752.GH11942@merlinux.eu>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
	<20120604142752.GH11942@merlinux.eu>
Message-ID: <CAGT4ZFj=Q4gQQAdGXJhwuDf=uR2ejJM5Bv1Yc89WcuCasnf-Dw@mail.gmail.com>

Hi there,

On Mon, Jun 4, 2012 at 4:27 PM, holger krekel <holger at merlinux.eu> wrote:

> I am wondering, however, how many people really have that py2/py3 need

I'm not sure that's precisely the right question, as you indicate
yourself already:

> OTOH,
> a good solution could trigger needs - that's how a lot of technology
> development works, anyway :)

Indeed. Right now many people are developing Python 2 projects. They
might be interested in a way to use Python 3 libraries in such a
project.

Right now some people are developing Python 3 projects. THey migh be
interested in a way to use Python 2 libraries in such a project. This
might in fact motivate more people to start a Python 3 project in the
first place, as they aren't faced with having to port any libraries
they may want to use.

If you asked these people a question "would you like to be able to use
Python 2 libraries in Python 3 projects" (or reversed) many of them
might say 'yes'.

But none of these people may feel they *need* a py2/py3 interpreter. :)

> and how much they care about performance in such situations.

Yes, such people might say "yes, if there isn't any other impact on my
project" (like performance). Of course there are other consequences to
adopting PyPy in a project, still..

Bringing up this idea once every while is my little contribution to
the Python 3 effort. At present I don't want to go any further. :)

Regards,

Martijn

From tismer at stackless.com  Mon Jun  4 21:36:53 2012
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 04 Jun 2012 21:36:53 +0200
Subject: [pypy-dev] change of strategy for the py3k branch?
In-Reply-To: <4FC72C00.4000508@gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGmFidZ1JK=oKpJZbOH0LBxmkyM51vH=AB8EM1vY+J6OQkNW3Q@mail.gmail.com>
	<4FC5EB87.3030803@gmail.com>
	<CAGmFidYSsJ3iwd2WYxkA9f5_O2bw_OkVG8RNxcXBx7s3bxv4mw@mail.gmail.com>
	<4FC60614.4030505@gmail.com>
	<CAMSv6X1ehfwYXs7wqZoX1E9BXaFA1id-y_RWPcRM6Rs5TR6k8Q@mail.gmail.com>
	<4FC72C00.4000508@gmail.com>
Message-ID: <4FCD0E55.9070900@stackless.com>

On 5/31/12 10:29 AM, Antonio Cuni wrote:
> On 05/30/2012 04:04 PM, Armin Rigo wrote:
>
>> Then we can probably arrange things so that we use "translate.py" from
>> default, and not from the "py3k branch", which would be stripped of
>> the translation parts.
>>
>> More precisely, we could organize this "py3k branch" --- quotes,
>> because likely living then in another repo --- with an only marginally
>> different directory structure: e.g. call the top-level directory
>> "py3k" instead of "pypy".  Then you would use the default's
>> "translate.py" to translate it, without getting conflicts between
>> "pypy.interpreter" as used by translate.py and the new
>> "py3k.interpreter" containing what you are translating.
>>
>> Of course the directories that would be in the py3k package would
>> still have the same name as their original ones, so that we keep open
>> the possibility to do merges without adding yet another layer of
>> troubles.
> uhm, that's an interesting possibility, I didn't think of it. I wonder if
> mercurial handles merges well if we rename the top-level directory.
>
> To make things cleaner and easier to understand, we should probably also "hg
> rm" from py3k/ the directories which belongs to the toolchain, just to avoid
> confusion. I think that in this case at each merge mercurial would ask what to
> do with file X which has been deleted locally but changed remotely, but this
> is probably something that we can handle.
>
> As I said earlier, the drawback of such "decoupling" solutions is that as soon
> as you have two separate repos, you'll get troubles such as "you can translate
> revision XXX only if the pypy repo is at version YYY", which can be
> frustrating especially when you want to go back in the history.
>
> In theory mercurial subrepos are supposed to solve this problem, but in
> practice we should stay as far as we can from them :-(.
>
> Amaury: opinions on Armin's proposed solution?
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

Hi,

I really have to second that last statement:
Subrepos are a red herring for me, after having lots of trouble
in our much smaller project..
The idea is qood, but not in the next half or one year at least.

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key ->  http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/


From tismer at stackless.com  Mon Jun  4 22:04:57 2012
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 04 Jun 2012 22:04:57 +0200
Subject: [pypy-dev] connecting multiple interpreters
In-Reply-To: <CAKCKLWwpeKBOtLVSFX=Owi=w67YDCTPX9sYjPue-E_BjEdgp8A@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
	<CAKCKLWwpeKBOtLVSFX=Owi=w67YDCTPX9sYjPue-E_BjEdgp8A@mail.gmail.com>
Message-ID: <4FCD14E9.7020009@stackless.com>

On 6/4/12 4:26 PM, Michael Foord wrote:
>
>
> On 4 June 2012 14:33, Armin Rigo <arigo at tunes.org 
> <mailto:arigo at tunes.org>> wrote:
>
>     Hi Martijn, hi Holger,
>
>     On 6/1/12, Martijn Faassen <faassen at startifact.com
>     <mailto:faassen at startifact.com>> wrote:
>     > Concerning performance overhead of proxies, lifecycle issues
>     would be
>     > tricky
>
>     If, hypothetically speaking, there is someone interested in writing a
>     PyPy solution where both a Python2 and a Python3 interpreter are
>     running in the same process, then you gain the advantage of having
>     only one GC to run both.  At least it transparently solves the issues
>     of lifetime and reference cycles.  (You also have for free only one
>     JIT, which can do cross-language optimizations like inlining a Python2
>     function into a Python3 context or vice-versa).  I see these two
>     points as benefits that you don't have in any multi-process solution.
>
>     It would require some work on the PyPy side, and I'm not aware of
>     anybody ready to invest time in that, but it shouldn't be particularly
>     hard (once PyPy's Python3 interpreter is ready, and once people agree
>     about which API to use to do cross-language calls.)
>
>
>
>
> Having multiple interpreter instances within a single process allows 
> for lots of interesting possibilities. IronPython permits this and it 
> was used  by Resolver One - just one of the reasons (along with the 
> GIL) that it would have been much harder to write Resolver One in 
> CPython than in IronPython (although .NET was chosen as a platform 
> before Python was chosen as an implementation language).
>
> Allowing Python 2 and Python 3 to live within the same process would 
> be very interesting.

I too like this idea very much, because _that_ removes the need for a 
decision which
I really can't base upon taste, but module availabilities, 
PyPy-readiness etc.
Such a bridge would be very cool and quite 'peace-making' if may say so.

When thinking of it, a slightly crazy concern popped up:
I just followed the pyvenv discussions. PEP 405 will not be back-ported
to Python2.7.
If we now have two interpreter versions in one binary, which one of
the Janus-heads will lead decisions like pyvenv startup?

just-semi-seriously ;-) -- chris

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key ->  http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/968f8e3b/attachment-0001.html>

From amauryfa at gmail.com  Mon Jun  4 22:09:59 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Jun 2012 22:09:59 +0200
Subject: [pypy-dev] connecting multiple interpreters
In-Reply-To: <4FCD14E9.7020009@stackless.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
	<CAKCKLWwpeKBOtLVSFX=Owi=w67YDCTPX9sYjPue-E_BjEdgp8A@mail.gmail.com>
	<4FCD14E9.7020009@stackless.com>
Message-ID: <CAGmFidaSKir+mAfD7fWV-EP5YxYRdjONCvahfHgJueDdo1-1Bg@mail.gmail.com>

2012/6/4 Christian Tismer <tismer at stackless.com>

> When thinking of it, a slightly crazy concern popped up:
> I just followed the pyvenv discussions. PEP 405 will not be back-ported
> to Python2.7.
> If we now have two interpreter versions in one binary, which one of
> the Janus-heads will lead decisions like pyvenv startup?
>

Even if there are two interpreters, only one will run at start-up,
modules from the other interpreter need a special "import_py3k" function.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/736365f8/attachment.html>

From tismer at stackless.com  Mon Jun  4 22:22:14 2012
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 04 Jun 2012 22:22:14 +0200
Subject: [pypy-dev] connecting multiple interpreters
In-Reply-To: <CAGmFidaSKir+mAfD7fWV-EP5YxYRdjONCvahfHgJueDdo1-1Bg@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
	<CAKCKLWwpeKBOtLVSFX=Owi=w67YDCTPX9sYjPue-E_BjEdgp8A@mail.gmail.com>
	<4FCD14E9.7020009@stackless.com>
	<CAGmFidaSKir+mAfD7fWV-EP5YxYRdjONCvahfHgJueDdo1-1Bg@mail.gmail.com>
Message-ID: <4FCD18F6.1090800@stackless.com>

On 6/4/12 10:09 PM, Amaury Forgeot d'Arc wrote:
> 2012/6/4 Christian Tismer <tismer at stackless.com 
> <mailto:tismer at stackless.com>>
>
>     When thinking of it, a slightly crazy concern popped up:
>     I just followed the pyvenv discussions. PEP 405 will not be
>     back-ported
>     to Python2.7.
>     If we now have two interpreter versions in one binary, which one of
>     the Janus-heads will lead decisions like pyvenv startup?
>
>
> Even if there are two interpreters, only one will run at start-up,
> modules from the other interpreter need a special "import_py3k" function.
>

Sure, I was just kidding. But if the py3k support is very marginal,
just enough to start an imported py2.7 interpreter, then this would
be a way to have pyvenv running with Python2.7.

ok I'll shut up/down now - chris

-- 
Christian Tismer             :^)<mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key ->  http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/bb90adc5/attachment.html>

From amauryfa at gmail.com  Mon Jun  4 22:32:55 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Jun 2012 22:32:55 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
Message-ID: <CAGmFidZuwP4z=AoXRJD+mnU-bO0Y97d5CtUk12pr7RqeADPzVA@mail.gmail.com>

Hi,

2012/6/4 Armin Rigo <arigo at tunes.org>

> Hi Martijn, hi Holger,
>
> On 6/1/12, Martijn Faassen <faassen at startifact.com> wrote:
> > Concerning performance overhead of proxies, lifecycle issues would be
> > tricky
>
> If, hypothetically speaking, there is someone interested in writing a
> PyPy solution where both a Python2 and a Python3 interpreter are
> running in the same process, then you gain the advantage of having
> only one GC to run both.  At least it transparently solves the issues
> of lifetime and reference cycles.  (You also have for free only one
> JIT, which can do cross-language optimizations like inlining a Python2
> function into a Python3 context or vice-versa).  I see these two
> points as benefits that you don't have in any multi-process solution.
>
> It would require some work on the PyPy side, and I'm not aware of
> anybody ready to invest time in that, but it shouldn't be particularly
> hard (once PyPy's Python3 interpreter is ready, and once people agree
> about which API to use to do cross-language calls.)
>

I did some experiments a few months ago, trying to embed two 2.7
interpreters in the same translation target
(which could have slightly different options, say with/without rope
strings).

I stopped when I realized that it cannot work with a single pypy tree: if
space.get_name() returns a RPython string in 2.7, it cannot return a
RPython unicode in 3.2; and even then, the MethodOfFrozenPBCRepr class
"assumes that all methods are the same function bound to different PBCs".

Maybe we should retry with different pypy trees (i.e pypy/interpreter/ vs.
pypy3k/interpreter/, and so on)
But I'm sure we would like some objects to be compatible, for
example PyFrame2.f_backref could be a PyFrame3.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/075cc504/attachment.html>

From wlavrijsen at lbl.gov  Mon Jun  4 23:03:11 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 4 Jun 2012 14:03:11 -0700 (PDT)
Subject: [pypy-dev] reusing test classess from another module's test
	directory?
Message-ID: <alpine.LNX.2.00.1206041351480.1119@pc3-wlav.lbl.gov>

Hello,

in the PyObject* converter and executor, I use cpyext to get hold of a
PyObject* to send to/receive from the C++ world. That works okay, but it
proves rather hard to test.

I now have a test that does what I want, based on AppTestCpythonExtensionBase,
where pretty much all the heavy lifting is done, saving me tons of work.

What is the recommended way to grab test bases from the test area of another
module? Right now, I'm doing the moderately ugly:

   sys.path.append(str(py.path.local(autopath.pypydir).join('module', 'cpyext', 'test')))
   from test_cpyext import AppTestCpythonExtensionBase

which works fine, but surely there must be a better way?

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From amauryfa at gmail.com  Mon Jun  4 23:09:17 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Jun 2012 23:09:17 +0200
Subject: [pypy-dev] reusing test classess from another module's test
	directory?
In-Reply-To: <alpine.LNX.2.00.1206041351480.1119@pc3-wlav.lbl.gov>
References: <alpine.LNX.2.00.1206041351480.1119@pc3-wlav.lbl.gov>
Message-ID: <CAGmFidZ33+CzgJurVSkLwOddKG1+4JKkUrpLg1LA5HMmCEc09g@mail.gmail.com>

2012/6/4 <wlavrijsen at lbl.gov>

> What is the recommended way to grab test bases from the test area of
> another
> module? Right now, I'm doing the moderately ugly:
>
>  sys.path.append(str(py.path.**local(autopath.pypydir).join('**module',
> 'cpyext', 'test')))
>  from test_cpyext import AppTestCpythonExtensionBase
>
> which works fine, but surely there must be a better way?
>

You surely noticed that pypy uses absolute imports everywhere.
Did you try:
    from pypy.module.cpyext.test.test_cpyext import
AppTestCpythonExtensionBase
like all test files under pypy/module/cpyext?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/950887f1/attachment-0001.html>

From wlavrijsen at lbl.gov  Mon Jun  4 23:15:02 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 4 Jun 2012 14:15:02 -0700 (PDT)
Subject: [pypy-dev] reusing test classess from another module's test
 directory?
In-Reply-To: <CAGmFidZ33+CzgJurVSkLwOddKG1+4JKkUrpLg1LA5HMmCEc09g@mail.gmail.com>
References: <alpine.LNX.2.00.1206041351480.1119@pc3-wlav.lbl.gov>
	<CAGmFidZ33+CzgJurVSkLwOddKG1+4JKkUrpLg1LA5HMmCEc09g@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1206041413460.1119@pc3-wlav.lbl.gov>

Hi Amaury,

> You surely noticed that pypy uses absolute imports everywhere.
> Did you try:
>    from pypy.module.cpyext.test.test_cpyext import AppTestCpythonExtensionBase
> like all test files under pypy/module/cpyext?

yes, now I see. Before, I overlooked that directory 'test' actually has an
__init__.py, thus being a module as well.

Thanks,
      Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From wlavrijsen at lbl.gov  Tue Jun  5 01:53:45 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Mon, 4 Jun 2012 16:53:45 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <2785332.RJWjHmEeLe@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
	<2785332.RJWjHmEeLe@hunter-laptop>
Message-ID: <alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>

Hi Alex,

> Okay, with respect to the SWIG + pypy, works for me, and about 4x slower then
> cppyy or cpython ( which show same results more or less).

I'll try the SWIG portion later (as well as all the other points that you
raise in this thread; I think the feedback is great). First, I now have the
PyObject* tests (whew!) and that has allowed me to push the code.

Also, I found one reason why my micro-benchmark was so slow: the objects were
all kept alive (ref counting problem), meaning the number of objects to keep
track of by cpyext kept growing, thus the lookups became slower and slower.
The tests caught that. :) With that solved, micro-bench speeds are okay-ish.

Then, rewriting your example to use PyObject* directly, there is one problem:
the C++ side now needs an additional Py_IncRef (which is a function call, not
a macro, in PyPy, adding extra overhead) since it takes ownership. Keeping
the python objects alive on the python side does not work as well, since e.g.
ints are apparently not pinned down, like I thought earlier, so you may get a
fresh one with the same value on the C++ side.

Note that if you use the typedef ref_t, a selection file is needed to tell
Reflex about that typedef, or no binding will be generated for it. Alternative,
you can use PyObject* directly (which is also a typedef, but I hard-coded that
name in cppyy, together with the actual _object, for convenience).

Okay, so speeds ... It's not too bad. As discussed before, your method and
the cpyext one are mostly the same, so similar should be expected, and it is:

CPPYY
Created container
No more items

real    0m2.013s
user    0m1.931s
sys     0m0.073s

Cpython
Created container
No more items

real    0m2.722s
user    0m2.556s
sys     0m0.157s

I then tried with the fast path enabled, but clearly all the time is being
spent in cpyext, so I although it does show consistent better timings, the
difference is only 0.02 secs. Maybe that could be faster, but cpyext is an
impressive piece of work, quite a bit above my PyPy-skills. :}

Overall, I'm happy enough with the result (I'm going to need handling of
PyObject*'s later for my own nefarious purposes).

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From rorsoft at gmail.com  Tue Jun  5 03:46:12 2012
From: rorsoft at gmail.com (bookaa)
Date: Tue, 5 Jun 2012 09:46:12 +0800
Subject: [pypy-dev] MinGW32 support PyPy with mscr90.dll
Message-ID: <509BC4043C474AD595CBCE3792F0BF1F@vSHliutaotao>

Windows system, after 
    set cc=mingw32-gcc
PyPy can not pass many tests:
    pytest.py pypy/translator/test/test_unsimplify.py 
    pytest.py pypy/translator/c/test/test_extfunc.py::test_dictlike_environ_getitem

    ...
I find this is because MinGW32 gcc compile default use msvcrt.dll, but we need msvcr90.dll,
as Python 2.7 use msvcr90.dll.
  >>> import ctypes
  >>> ctypes.util.find_msvcrt()
  'msvcr90.dll'And even python.exe dependence on msvcr90.dllref to http://trac.seqan.de/wiki/HowTo/SelectASpecificMsvcrtVersionWithMinGw
this is how I make MinGW32 gcc use msvcr90.dll instead:


save these to you MinGW directory as file D:\MinGW\lib\gcc\mingw32\4.6.2\specs


  *cpp:
  %{posix:-D_POSIX_SOURCE} %{mthreads:-D_MT} -D__MSVCRT_VERSION__=0x0900


  *libgcc:
  %{mthreads:-lmingwthrd} -lmingw32 %{shared-libgcc:-lgcc_s} %{!shared-libgcc:-lgcc_eh} -lgcc -lmoldname90 -lmingwex -lmsvcr90


I suggest this instruction should be add to PyPy doc: MinGW32 support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120605/883dede7/attachment.html>

From nikuda at gmail.com  Tue Jun  5 05:13:44 2012
From: nikuda at gmail.com (ante)
Date: Tue, 5 Jun 2012 13:13:44 +1000
Subject: [pypy-dev] New pypy website update
In-Reply-To: <CAK5idxROG77YFt+Pt_SJrfD_fCpZ-_J8aT6jDDcQDvRXbPiyEg@mail.gmail.com>
References: <CAPPs_vCyd1hwvvJ5Fo6Pu=yKpRG5aUJ_-6UXJ_xcVPpQxSYVEA@mail.gmail.com>
	<CAFRnB2X76seyuALnri68pdZVEaEswzP1_LSadZ_XC4RXqbNEWg@mail.gmail.com>
	<CAK5idxROG77YFt+Pt_SJrfD_fCpZ-_J8aT6jDDcQDvRXbPiyEg@mail.gmail.com>
Message-ID: <CAPPs_vCHVsxDosxWO1ASp32hvaVy8-L9A3oMvbw0ugPu9AXW_g@mail.gmail.com>

Not a problem, there's no rush. Whenever you get a chance we can go over
some of the details.

Thanks,

On Sat, Jun 2, 2012 at 9:09 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:

> On Sat, Jun 2, 2012 at 4:40 AM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>
>>
>>
>> On Fri, Jun 1, 2012 at 9:24 PM, ante <nikuda at gmail.com> wrote:
>>
>>>  Hi all,
>>>
>>> Here are some of the major updates done to the new website in recent
>>> months.
>>>
>>> - Download links automatically updated from bitbucket/pypy download page.
>>>  - Supported libs automatically updated from bitbucket/pypy wiki.
>>> - Fixed usability/linking issues with the sub section navigation.
>>> - Code highlighting
>>> - Write content in either ReST/raw text/HTML.
>>> - Working twitter feed.
>>> - Better performance overall.
>>> - Migrated from ep.io to dotCloud.
>>>
>>> www: http://pypy-nikuda.dotcloud.com/
>>> hg: https://bitbucket.org/pypy/pypy-site
>>> wiki: https://bitbucket.org/nikuda/pypy-site/wiki/Home
>>>
>>> To address some of the issues brought up last time the new website was
>>> discussed: One of the reasons for updating the design of the website was to
>>> make it more accessible to end-users, while having the deeper technical
>>> details available for those that need it. This is the reason behind
>>> implementing the 'hidden' until wanted, or dig down approach in the
>>> structure of the website. I understand this can seem unnecessary to people
>>> who are very used to the project and how it works. Even though I don't see
>>> a problem with it, it's only one possible solution and we can change things
>>> around if we agree that this particular implementation is not suited.
>>>
>>> The website is not finished and there are a few things that I want to
>>> change with it, but I do believe there's a good working base there that can
>>> be expanded on and improved. Approach is definitively an iterative one, so
>>> feel free add suggestions here or in the wiki.
>>>
>>> Thanks,
>>> Ante
>>>
>>> _______________________________________________
>>> pypy-dev mailing list
>>> pypy-dev at python.org
>>> http://mail.python.org/mailman/listinfo/pypy-dev
>>>
>>>
>> Hey, I haven't had a chance to look everything over, but I wanted to
>> thank you for all your hard work on this!
>>
>> Alex
>>
>> Same here. I'm sorry we haven't been responsive enough (in fact your
> mails are the oldest starred mails in my inbox).
>
> Thanks for all the hard work, let's see what we can do!
>
> Cheers,
> fijal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120605/9fbae4b1/attachment-0001.html>

From arigo at tunes.org  Tue Jun  5 09:43:24 2012
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 5 Jun 2012 09:43:24 +0200
Subject: [pypy-dev] connecting multiple interpreters (was: Re: change of
 strategy for the py3k branch?)
In-Reply-To: <CAGmFidZuwP4z=AoXRJD+mnU-bO0Y97d5CtUk12pr7RqeADPzVA@mail.gmail.com>
References: <4FC5D552.9050002@gmail.com>
	<CAGT4ZFhPjJYqdaXFO=Oq=9S7_BvcRbuShu149m5XovLAhZKJmA@mail.gmail.com>
	<20120531091217.GI10174@merlinux.eu>
	<CAGT4ZFhjP1p+5qGiYbC_rDpn81o2So6saqHeFCP_2W_GTvsOnQ@mail.gmail.com>
	<20120531211809.GP10174@merlinux.eu>
	<CAGT4ZFiiR88ChMHUPfR=+WOwUhX=fpSCxfqUA=9wcb6EdGHcCw@mail.gmail.com>
	<CAMSv6X1b_kc9YRxZF3+wnS94aunE1pTOz_V-Mfj7qf8m59AGPw@mail.gmail.com>
	<CAGmFidZuwP4z=AoXRJD+mnU-bO0Y97d5CtUk12pr7RqeADPzVA@mail.gmail.com>
Message-ID: <CAMSv6X0v5aRaoz9i5z0thYM3Ve9XQgoVyhoPjSL4NgnS_sGeLQ@mail.gmail.com>

Hi Amaury,

On Mon, Jun 4, 2012 at 10:32 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> Maybe we should retry with different pypy trees (i.e pypy/interpreter/ vs.
> pypy3k/interpreter/, and so on)
> But I'm sure we would like some objects to be compatible, for
> example?PyFrame2.f_backref could be a PyFrame3.

Not necessarily.  At least not if the API presented to the user is
that of a foreign function invocation library.  We'd start with a
no-sharing model and focus on enabling some data exchanges beyond the
minimal "foreign data" boxes in both directions.


A bient?t,

Armin.

From arigo at tunes.org  Tue Jun  5 09:54:25 2012
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 5 Jun 2012 09:54:25 +0200
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
	<2785332.RJWjHmEeLe@hunter-laptop>
	<alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
Message-ID: <CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>

Hi Wim,

On Tue, Jun 5, 2012 at 1:53 AM,  <wlavrijsen at lbl.gov> wrote:
> Overall, I'm happy enough with the result (I'm going to need handling of
> PyObject*'s later for my own nefarious purposes).

Maybe we can motivate you to look for and fix inefficiencies in
cpyext :-)  As far as I know it's still fully designed for "it works
and we don't care about speed".  See for example
https://bugs.pypy.org/issue1121 which tries to push optimizations in
cpyext but with no-one really interested from the core group.

But IMHO the main slow part that should be optimized (probably by
being rewritten from scratch) is the correspondance between the proxy
PyObject structures and the real PyPy objects, which is exactly the
part that interests you too.  It should be possible at least to cope
with only one dict lookup instead of at least two to do the roundtrip
"pypy obj -> cpyext obj -> pypy obj".


A bient?t,

Armin.

From alex.pyattaev at gmail.com  Wed Jun  6 00:06:32 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Wed, 06 Jun 2012 01:06:32 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
	<CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
Message-ID: <6544216.odt3ZuislU@hunter-laptop>

Well, as far as I can see from where I am, the only real good that can come 
from interfacing with PyObject is the capability to write container classes in 
C++. On the other hand, it seems that it is not very often needed. For the 
case when objects just need to be LINKED from the C++ code as references, pure 
python/Rpython abstraction layer that would keep a backup copy of the object 
(which can be discarded from the python side) seems like a better idea. The 
main benefit is that there will be no refcounting problem and no slate 
pointers - if C++ returns some key that is not available then exception can be 
raised instead of segfault as with PyObject*. Also, no extra headers will be 
needed on C++ side, and the datatype can be an integer. Why not? As far as I 
understand, modifying python objects in C++ with pypy is massively stupid. 

By the way - the only reason i use C++ in my project is because I have a 
massive amount of legacy code that uses several C libraries and Qt, as well as 
openGL, so porting that bunch to python can be quite a hassle. The only 
problem is that Python side has to send certain requests to C++, those should 
be cached, processed and reported back up when done. Obviously, request object 
itself could travel along with the request data for code clarity, but there 
can just as well be an int. The only problem with ints is the lifetime of the 
references in the mapping structure, which would have to be managed explicitly 
in such case. If you are interested, I could send you some thoughts on that 
issue specifically, as I was planning to do this kind of thing at some point.

BR,
Alex.
tiistai 05 kes?kuu 2012 09:54:25 Armin Rigo kirjoitti:
> Hi Wim,
> 
> On Tue, Jun 5, 2012 at 1:53 AM,  <wlavrijsen at lbl.gov> wrote:
> > Overall, I'm happy enough with the result (I'm going to need handling of
> > PyObject*'s later for my own nefarious purposes).
> 
> Maybe we can motivate you to look for and fix inefficiencies in
> cpyext :-)  As far as I know it's still fully designed for "it works
> and we don't care about speed".  See for example
> https://bugs.pypy.org/issue1121 which tries to push optimizations in
> cpyext but with no-one really interested from the core group.
> 
> But IMHO the main slow part that should be optimized (probably by
> being rewritten from scratch) is the correspondance between the proxy
> PyObject structures and the real PyPy objects, which is exactly the
> part that interests you too.  It should be possible at least to cope
> with only one dict lookup instead of at least two to do the roundtrip
> "pypy obj -> cpyext obj -> pypy obj".
> 
> 
> A bient?t,
> 
> Armin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120606/5e3fa8c6/attachment.html>

From wlavrijsen at lbl.gov  Wed Jun  6 00:59:57 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Tue, 5 Jun 2012 15:59:57 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <6544216.odt3ZuislU@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
	<CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
	<6544216.odt3ZuislU@hunter-laptop>
Message-ID: <alpine.LNX.2.00.1206051554400.1119@pc3-wlav.lbl.gov>

Hi Alex,

> By the way - the only reason i use C++ in my project is because I have a
> massive amount of legacy code that uses several C libraries and Qt, as well as
> openGL, so porting that bunch to python can be quite a hassle. The only
> problem is that Python side has to send certain requests to C++, those should
> be cached, processed and reported back up when done.

somehow I have a feeling that here's a general use case in there that should
be supported in a more direct way, but I can't quite put my finger on it. One
thing I'm missing, for example, is: what is doing the unpacking of the python
objects (requests) into something that the C++ side understands?

Another option may be that if the PyObject*'s can be reused (e.g. if there is
a fixed set of kinds of requests), the overhead is much, much less, as the JIT
can then consider them constants.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From wlavrijsen at lbl.gov  Wed Jun  6 09:10:36 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Wed, 6 Jun 2012 00:10:36 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
	<2785332.RJWjHmEeLe@hunter-laptop>
	<alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
	<CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
Message-ID: <alpine.LNX.2.00.1206052356010.10553@pc3-wlav.lbl.gov>

Hi Armin,

> But IMHO the main slow part that should be optimized (probably by
> being rewritten from scratch) is the correspondance between the proxy
> PyObject structures and the real PyPy objects, which is exactly the
> part that interests you too.  It should be possible at least to cope
> with only one dict lookup instead of at least two to do the roundtrip
> "pypy obj -> cpyext obj -> pypy obj".

I'll need to understand first in more detail how cpyext works, but it is
good motivation to know that a dict lookup can be removed. In cppyy, I
have a similar, but reversed, problem: keeping track of all C++ objects
to guarantee object identity. Since the bound object carries a pointer
to the C++ object, a round trip has only one dict lookup, the other
"lookup" being a data member access.

If I understand correctly what you describe above, then perhaps giving
the "PyPyObject by pointer" as payload to the PyObject could help, too.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From arigo at tunes.org  Wed Jun  6 11:00:10 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 6 Jun 2012 11:00:10 +0200
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206052356010.10553@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<10542917.ncyhjCfde6@hunter-laptop>
	<alpine.LNX.2.00.1206031549420.28010@pc3-wlav.lbl.gov>
	<2785332.RJWjHmEeLe@hunter-laptop>
	<alpine.LNX.2.00.1206041555320.1119@pc3-wlav.lbl.gov>
	<CAMSv6X2TkwQ-8vpTZ0Qihn_psAcqCY_itTKZBquY+D7h8CKytA@mail.gmail.com>
	<alpine.LNX.2.00.1206052356010.10553@pc3-wlav.lbl.gov>
Message-ID: <CAMSv6X006fETgBmg-bSADU0Uw5J5v50vRzcoP4y-qkP2idkjoA@mail.gmail.com>

Hi Wim,

On Wed, Jun 6, 2012 at 9:10 AM,  <wlavrijsen at lbl.gov> wrote:
> If I understand correctly what you describe above, then perhaps giving
> the "PyPyObject by pointer" as payload to the PyObject could help, too.

Yes: so far, most PyObjects are small structures containing just the
refcount and the cpyext-ified type.  Note that for your use case you
don't need the latter, which would simplify things a lot.  If the goal
is "only" to pass around PyObjects as black boxes, as fast as
reasonably possible, then the easiest route would be to not use cpyext
at all, but instead try to come up with your own mapping logic.  This
looks quite easier for experimentation purposes.  And if you end up
with a good and fast solution we can always try later to reuse it in
cpyext.

Trying to think about the API to use: there is
pypy.rlib.rgc.malloc_nonmovable(TP), where TP is a GcStruct.  It
allocates a normal GC object but tries to make sure that it will never
move.  Raw pointers to that can be passed around to C++, but won't
keep the GcStruct alive by themselves, of couse.  I'm not sure if this
is the best suited API; maybe we can discuss and make a better one
too.  For example:

1) a pair of functions to register and unregister a GC pointer that
lives outside the GC

2) or maybe a way to register and unregister "external" GcStructs:
you'd still allocate and free raw bytes for the PyObject wrapper, but
making sure it has a GcStruct shape (i.e. some standard GC headers).
You would then register and unregister it to be considered by the GC.
This looks easy to implement: it is the same as we consider all the
prebuilt GcStructs that exist at translation time.  In other words,
they are a source of further GC pointers, but which live themselves
outside the scope managed by the GC.  The difference with 1) is that
the GC can still write flags in the header of these objects; it helps
a lot generational GCs.


A bient?t,

Armin.

From alex.pyattaev at gmail.com  Wed Jun  6 17:04:05 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Wed, 06 Jun 2012 19:04:05 +0400
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <CAMSv6X006fETgBmg-bSADU0Uw5J5v50vRzcoP4y-qkP2idkjoA@mail.gmail.com>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1206052356010.10553@pc3-wlav.lbl.gov>
	<CAMSv6X006fETgBmg-bSADU0Uw5J5v50vRzcoP4y-qkP2idkjoA@mail.gmail.com>
Message-ID: <29110551.LJ1cAUrdgA@hunter-laptop>

>  If the goal
> is "only" to pass around PyObjects as black boxes, as fast as
> reasonably possible, then the easiest route would be to not use cpyext
> at all, but instead try to come up with your own mapping logic.  This
> looks quite easier for experimentation purposes.  And if you end up
> with a good and fast solution we can always try later to reuse it in
> cpyext.
This is precisely the goal for me. The requests in my system are basically a 
C++ classes (structs, essentially), that are created and submitted to C++ core 
as pointers, they also hold a pointer to the PyObject that submitted the 
request. When the request is served, the pyobject can be notified about that 
without having to go through any sort of lookups. 
> 
> 1) a pair of functions to register and unregister a GC pointer that
> lives outside the GC
That might actually be a good idea. In that case a certain object can tell GC 
that it wants to have a valid external pointer, and when the object is 
destroyed, a __del__ function can remove that registration with GC. If anyone 
uses that pointer afterwards it is probably his/her own problem.

I think also that SWIG's idea with c++ side "owning" python objects is rather 
lame. Basically, if an object is owned by c++ and not cleaned properly all 
kinds of strange things can happen...

BR,
Alex.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120606/d3b197f6/attachment.html>

From matti.picus at gmail.com  Wed Jun  6 18:14:08 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Wed, 06 Jun 2012 19:14:08 +0300
Subject: [pypy-dev] support for ordinals on windows in _ffi and ctypes
Message-ID: <4FCF81D0.3040300@gmail.com>

I have started a branch to support app level finding functions by 
ordinals: win-ordinal

I would be happy for any comments as it progresses. Currently, I have 
added tests and ordinal handling to rlib/libffi.py and module/_ffi

if I run tests one at a time, they all pass, but running in one pass 
makes test_ztranslation fail, here is the traceback 
https://gist.github.com/2882909. Any thoughts?

Matti



From anto.cuni at gmail.com  Thu Jun  7 11:01:57 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 07 Jun 2012 11:01:57 +0200
Subject: [pypy-dev] support for ordinals on windows in _ffi and ctypes
In-Reply-To: <4FCF81D0.3040300@gmail.com>
References: <4FCF81D0.3040300@gmail.com>
Message-ID: <4FD06E05.4040604@gmail.com>

Hello Matti,

On 06/06/2012 06:14 PM, Matti Picus wrote:
> I have started a branch to support app level finding functions by ordinals:
> win-ordinal
> 
> I would be happy for any comments as it progresses. Currently, I have added
> tests and ordinal handling to rlib/libffi.py and module/_ffi

I read the commits and they look fine.

> if I run tests one at a time, they all pass, but running in one pass makes
> test_ztranslation fail, here is the traceback https://gist.github.com/2882909.
> Any thoughts?

If I had to dig into this, I'd probably start by trying to understand if there
is a test in particular which triggers this behavior.
However, 3f72e6d9820b by amaury claims to solve the issue. Not sure what's
going on, though.

ciao,
Anto

From uwe_f_mayer at yahoo.com  Thu Jun  7 18:44:26 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Thu, 7 Jun 2012 09:44:26 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
Message-ID: <1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>

Armin,
thanks for merging in the patch. I have now cloned the latest version, and tried to compile under Cygwin. I ran into a couple more issues, attached is a follow-up patch. In parallel I had a longer conversation with Matti about potentially setting up a build slave for Pypy under Cygwin. Clearly some kind of testing is needed. Currently the Cygwin Pypy Python standalone version build with --opt=jit fails on os.fork() calls. The?Pypy Python version build with --opt=2 handles os.fork()?apparently just fine, on the other hand it cannot be used to build Pypy and simply exits without error message after platcheck_28. So the question is, do you have a regular build for Cygwin?
I have not fully checked into whether I could set up a build slave, but it's clear already that because of the long time it takes to run a build (6.5 hours), it would have to be a weekly, not nightly, setup. I am willing to pursue that if it would help the project.
--Uwe


________________________________
 From: Armin Rigo <arigo at tunes.org>
To: Uwe F. Mayer <uwe_f_mayer at yahoo.com> 
Cc: "pypy-dev at python.org" <pypy-dev at python.org> 
Sent: Sunday, June 3, 2012 2:57 AM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin
 
Hi Uwe,

On Tue, May 1, 2012 at 1:05 AM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> I edited / hacked the sources and managed to compile PyPy under Cygwin based
> on Python 2.6.7 and gcc-4.5.3. If you find this of interest, the patches are
> available at http://www.tux.org/~mayer/cygwin/pypy/.

Thanks!? I finally took the time to review and merge the patch.

I didn't include any change to lib-python: they are changes that look
like they should also be done to CPython..?? I may be mistaken, but it
looks like fixes for tests that would also fail on CPython+cygwin.? If
that's the case, then it should be reported and fixed in CPython.


A bient?t,

Armin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/a6fc1842/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pypy.patch.no-python
Type: application/octet-stream
Size: 3496 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/a6fc1842/attachment.obj>

From amauryfa at gmail.com  Thu Jun  7 19:46:39 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 7 Jun 2012 19:46:39 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
Message-ID: <CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>

2012/6/7 Uwe F. Mayer <uwe_f_mayer at yahoo.com>

> Currently the Cygwin Pypy Python standalone version build with --opt=jit
> fails on os.fork() calls.


fork() on Windows... I'm surprised it works at all!

You should probably read this page:
http://cygwin.com/cygwin-ug-net/highlights.html#ov-hi-process
specially the "DLL base address collisions", did you run rebaseall?
Is pypy linked with some non-cygwin load-time DLLs?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/8936b95b/attachment.html>

From uwe_f_mayer at yahoo.com  Thu Jun  7 21:21:24 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Thu, 7 Jun 2012 12:21:24 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
Message-ID: <1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>

The pypy-c executable is not a dll, and hence does not need to be rebased if I understand it correctly. However, just to follow up on this hunch, I rebased it by first running rebaseall -v on the complete Cygwin installation (which does not touch pypy-c even if put into /usr/bin/) and then manually rebasing pypy-c to the next available base address. Unfortunately this does not help.

As to the question about runtime libraries, pypy links against the following libraries:

$ cygcheck ./pypy-c
D:\pypy\pypy-work-no-python-patch\pypy\pypy\translator\goal\pypy-c
? C:\cygwin\bin\cygbz2-1.dll
? ? C:\cygwin\bin\cygwin1.dll
? ? ? C:\windows\system32\KERNEL32.dll
? ? ? ? C:\windows\system32\ntdll.dll
? ? C:\cygwin\bin\cyggcc_s-1.dll
? C:\cygwin\bin\cygcrypt-0.dll
? C:\cygwin\bin\cygcrypto-1.0.0.dll
? ? C:\cygwin\bin\cygz.dll
? C:\cygwin\bin\cygncurses-10.dll
? C:\cygwin\bin\cygexpat-1.dll
? C:\cygwin\bin\cygintl-8.dll
? ? C:\cygwin\bin\cygiconv-2.dll
? C:\cygwin\bin\cygssl-1.0.0.dll
? C:\cygwin\bin\cygffi-4.dll

Just for good measure, here's a typical error message from an interactive pypy session:

$ pypy-c
Python 2.7.2 (b0005ab7c34f+, Jun 06 2012, 17:33:28)
[PyPy 1.9.1-dev0 with GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``'that's definitely a case of
"uh????"'''
>>>> import os
>>>> os.fork()
? ? ? 3 [main] pypy-c 11788 fixup_mmaps_after_fork: ReadProcessMemory failed for MAP_PRIVATE address 0xB0010000, Win32 error 998
? ? 153 [main] pypy-c 11788 D:\pypy\pypy-work-no-python-patch\pypy\pypy\translator\goal\pypy-c: *** fatal error in forked process - recreate_mmaps_after_fork_failed
? ? 499 [main] pypy-c 11788 open_stackdumpfile: Dumping stack trace to pypy-c.stackdump
? ? ? 2 [main] pypy-c 2572 fork: child -1 - forked process 11788 died unexpectedly, retry 0, exit code 256, errno 11
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last):
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? File "<console>", line 1, in <module>
? ? ? ? ? ? ? ? ? ? ? ? ? ? ?OSError: [Errno 11] Resource temporarily unavailable
?>>>>?


I noticed that the address 0xB0010000?in the error message above seems to be the same regardless of rebasing pypy-c or not, and I am seeing this address on two different Windows 2003 64bit servers. For what it's worth, the expected output, from a python session, would be:

$ python
Python 2.6.7 (r267:88850, Feb ?2 2012, 23:50:20)?
[GCC 4.5.3] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.fork()
5552
>>> 0
>>>?

Finally, today unexpectedly the os.fork() call works on a Windows XP Professional 32bit box, where it previously failed (that's the Cygwin pypy 1.8.1 version I had posted on my web page).

--Uwe


________________________________
 From: Amaury Forgeot d'Arc <amauryfa at gmail.com>
To: Uwe F. Mayer <uwe_f_mayer at yahoo.com> 
Cc: Armin Rigo <arigo at tunes.org>; Matti Picus <matti.picus at gmail.com>; "pypy-dev at python.org" <pypy-dev at python.org> 
Sent: Thursday, June 7, 2012 10:46 AM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin
 

2012/6/7 Uwe F. Mayer <uwe_f_mayer at yahoo.com>

Currently the Cygwin Pypy Python standalone version build with --opt=jit fails on os.fork() calls. 
fork() on Windows... I'm surprised it works at all!


You should probably read this page:

http://cygwin.com/cygwin-ug-net/highlights.html#ov-hi-process
specially the "DLL base address collisions", did you run rebaseall?
Is pypy linked with some non-cygwin load-time DLLs?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/fc33a8cd/attachment-0001.html>

From amauryfa at gmail.com  Thu Jun  7 23:24:46 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 7 Jun 2012 23:24:46 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
Message-ID: <CAGmFidZuZtZtBv_h8QqWpbd=wMMfRQeRvhkGkFWV-Vc4UpBjOw@mail.gmail.com>

2012/6/7 Uwe F. Mayer <uwe_f_mayer at yahoo.com>

> 3 [main] pypy-c 11788 fixup_mmaps_after_fork: ReadProcessMemory failed for
> MAP_PRIVATE address 0xB0010000, Win32 error 998


Aha, looks interesting. Searching for this error message, I found that
cygwin unblocks the parent process before the child calls
"fixup_mmaps_after_fork":
http://www.mail-archive.com/cygwin at cygwin.com/msg05547.html
Could you try again with strace?
Does the parent really unmap memory?
Or is it because of the special access rights that the JIT uses to mark the
memory as executable?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/63113140/attachment.html>

From wlavrijsen at lbl.gov  Fri Jun  8 00:34:33 2012
From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov)
Date: Thu, 7 Jun 2012 15:34:33 -0700 (PDT)
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <1543846.T9Sm08LvAA@hunter-laptop>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<alpine.LNX.2.00.1205301821010.1119@pc3-wlav.lbl.gov>
	<1543846.T9Sm08LvAA@hunter-laptop>
Message-ID: <alpine.LNX.2.00.1206071525290.1119@pc3-wlav.lbl.gov>

Hi Alex,

> With respect to build, I installed root from gentoo repository, and the
> problem there is that headers are in /usr/include/root and the actual libs are
> in /usr/lib/root, but the pypy build expects libs to be in
> /usr/include/root/lib if you specify SYSROOT=/usr/include/root.

now picking up include and libdirs via root-config, but then it does require
that tool to be available through PATH (it should be under $ROOTSYS/bin). If
not (e.g. standalone Reflex), it will still fall back to $ROOTSYS/include etc.

Best regards,
            Wim
-- 
WLavrijsen at lbl.gov    --    +1 (510) 486 6411    --    www.lavrijsen.net

From uwe_f_mayer at yahoo.com  Fri Jun  8 02:01:25 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Thu, 7 Jun 2012 17:01:25 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAGmFidZuZtZtBv_h8QqWpbd=wMMfRQeRvhkGkFWV-Vc4UpBjOw@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAGmFidZuZtZtBv_h8QqWpbd=wMMfRQeRvhkGkFWV-Vc4UpBjOw@mail.gmail.com>
Message-ID: <1339113685.38519.YahooMailNeo@web162905.mail.bf1.yahoo.com>

As requested, I ran the code with strace. More precisely, I ran from the pypy/pypy/translator/goal/ directory the command:
strace sh -c 'pypy-c /d/pypy/test_os_fork.py'
I ran this twice, once with a version of pypy built with "--opt=2", and once with?with a version of pypy built with?"--opt=jit". The?"--opt=2" version succeeds, and the "--opt=jit"?version fails, as reported earlier.
Attached you'll find the test_os_fork.py script, and the two log files cut down to the relevant part.?The log is a couple of MB, I cut it down to the relevant section.?
I don't know the answers the other two questions.

The third question may touch the core of the issue if the JIT marks the (parent) memory as exclusive access before the fork() call, which would be consistent with the observed error.?From what I can tell the fork() call never gets off the ground, having no read access to duplicate the process image.
--Uwe


________________________________
 From: Amaury Forgeot d'Arc <amauryfa at gmail.com>
To: Uwe F. Mayer <uwe_f_mayer at yahoo.com> 
Cc: Armin Rigo <arigo at tunes.org>; Matti Picus <matti.picus at gmail.com>; "pypy-dev at python.org" <pypy-dev at python.org> 
Sent: Thursday, June 7, 2012 2:24 PM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin
 

2012/6/7 Uwe F. Mayer <uwe_f_mayer at yahoo.com>

3 [main] pypy-c 11788 fixup_mmaps_after_fork: ReadProcessMemory failed for MAP_PRIVATE address 0xB0010000, Win32 error 998
Aha, looks interesting. Searching for this error message, I found that cygwin unblocks the parent process before the child calls "fixup_mmaps_after_fork":?http://www.mail-archive.com/cygwin at cygwin.com/msg05547.html
Could you try again with strace?
Does the parent really unmap memory?
Or is it because of the special access rights that the JIT uses to mark the memory as executable?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/1a7a8380/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: test_os_fork.py
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/1a7a8380/attachment-0001.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_os_fork_opt=2.log
Type: application/octet-stream
Size: 21270 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/1a7a8380/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_os_fork_opt=jit.log
Type: application/octet-stream
Size: 6164 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120607/1a7a8380/attachment-0003.obj>

From arigo at tunes.org  Fri Jun  8 07:27:53 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 8 Jun 2012 07:27:53 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
Message-ID: <CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>

Hi Uwe,

On Thu, Jun 7, 2012 at 9:21 PM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> ? ? ? 3 [main] pypy-c 11788 fixup_mmaps_after_fork: ReadProcessMemory failed
> for MAP_PRIVATE address 0xB0010000, Win32 error 998

Yes, that's one of the mmap addresses used by the JIT, if available.

More to the point, googling for "fork cygwin mmap" shows that there
are open bugs with cygwin's fork() in the presence of MAP_PRIVATE
mmaps.  That's obviously why it fails.

I don't really know what we can reasonably do about that, short of
asking people to use some specific versions of cygwin where it seems
to work.  I can imagine what we could unreasonably do as work-around,
but that requires a lot of work and gross hacks that don't seem
appropriate to put in the pypy source just to support cygwin...  I
would recommend that you try to see it fixed on the cygwin side.


A bient?t,

Armin.

From fijall at gmail.com  Fri Jun  8 11:50:08 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 8 Jun 2012 11:50:08 +0200
Subject: [pypy-dev] PyPy 1.9 - Yard Wolf
Message-ID: <CAK5idxSmpnbO1RnyGb9b815hqSS+cqrH+=G7=6a2Qxzw+keMxg@mail.gmail.com>

====================
PyPy 1.9 - Yard Wolf
====================

We're pleased to announce the 1.9 release of PyPy. This release brings
mostly
bugfixes, performance improvements, other small improvements and overall
progress on the `numpypy`_ effort.
It also brings an improved situation on Windows and OS X.

You can download the PyPy 1.9 release here:

    http://pypy.org/download.html

.. _`numpypy`: http://pypy.org/numpydonate.html


What is PyPy?
=============

PyPy is a very compliant Python interpreter, almost a drop-in replacement
for
CPython 2.7. It's fast (`pypy 1.9 and cpython 2.7.2`_ performance
comparison)
due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64 or
Windows 32.  Windows 64 work is still stalling, we would welcome a volunteer
to handle that.

.. _`pypy 1.9 and cpython 2.7.2`: http://speed.pypy.org


Thanks to our donors
====================

But first of all, we would like to say thank you to all people who
donated some money to one of our four calls:

  * `NumPy in PyPy`_ (got so far $44502 out of $60000, 74%)

  * `Py3k (Python 3)`_ (got so far $43563 out of $105000, 41%)

  * `Software Transactional Memory`_ (got so far $21791 of $50400, 43%)

  * as well as our general PyPy pot.

Thank you all for proving that it is indeed possible for a small team of
programmers to get funded like that, at least for some
time.  We want to include this thank you in the present release
announcement even though most of the work is not finished yet.  More
precisely, neither Py3k nor STM are ready to make it in an official release
yet: people interested in them need to grab and (attempt to) translate
PyPy from the corresponding branches (respectively ``py3k`` and
``stm-thread``).

.. _`NumPy in PyPy`: http://pypy.org/numpydonate.html
.. _`Py3k (Python 3)`: http://pypy.org/py3donate.html
.. _`Software Transactional Memory`: http://pypy.org/tmdonate.html

Highlights
==========

* This release still implements Python 2.7.2.

* Many bugs were corrected for Windows 32 bit.  This includes new
  functionality to test the validity of file descriptors; and
  correct handling of the calling convensions for ctypes.  (Still not
  much progress on Win64.) A lot of work on this has been done by Matti
Picus
  and Amaury Forgeot d'Arc.

* Improvements in ``cpyext``, our emulator for CPython C extension modules.
  For example PyOpenSSL should now work.  We thank various people for help.

* Sets now have strategies just like dictionaries. This means for example
  that a set containing only ints will be more compact (and faster).

* A lot of progress on various aspects of ``numpypy``. See the
`numpy-status`_
  page for the automatic report.

* It is now possible to create and manipulate C-like structures using the
  PyPy-only ``_ffi`` module.  The advantage over using e.g. ``ctypes`` is
that
  ``_ffi`` is very JIT-friendly, and getting/setting of fields is translated
  to few assembler instructions by the JIT. However, this is mostly intended
  as a low-level backend to be used by more user-friendly FFI packages, and
  the API might change in the future. Use it at your own risk.

* The non-x86 backends for the JIT are progressing but are still not
  merged (ARMv7 and PPC64).

* JIT hooks for inspecting the created assembler code have been improved.
  See `JIT hooks documentation`_ for details.

* ``select.kqueue`` has been added (BSD).

* Handling of keyword arguments has been drastically improved in the
best-case
  scenario: proxy functions which simply forwards ``*args`` and ``**kwargs``
  to another function now performs much better with the JIT.

* List comprehension has been improved.

.. _`numpy-status`: http://buildbot.pypy.org/numpy-status/latest.html
.. _`JIT hooks documentation`: http://doc.pypy.org/en/latest/jit-hooks.html

JitViewer
=========

There will be a corresponding 1.9 release of JitViewer which is guaranteed
to work with PyPy 1.9. See the `JitViewer docs`_ for details.

.. _`JitViewer docs`: http://bitbucket.org/pypy/jitviewer

Cheers,
The PyPy Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120608/167710c7/attachment.html>

From ryandrake at caryon.com  Fri Jun  8 19:19:46 2012
From: ryandrake at caryon.com (Ryan Drake)
Date: Fri, 8 Jun 2012 18:19:46 +0100
Subject: [pypy-dev] Placement- Site- Analysis
Message-ID: <16496995.20120608181946@caryon.com>

pypy-dev at codespeak.net    :   


Your site has a good look and feel but you're not getting as much web traffic as you could be getting.  Can we show you how to change that?  It won't cost you anything for us to review your online business and the results could spike your web revenue.  Reply to us today and we will do a free assessment of your web presence.  Don't forget to include the best way we can contact you.
Sincerely,

Ryan Drake
  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120608/99c0b07b/attachment.html>

From arigo at tunes.org  Sat Jun  9 11:22:49 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 9 Jun 2012 11:22:49 +0200
Subject: [pypy-dev] update (+patch) on embedding pypy
In-Reply-To: <df4204f44f9dbb996797e312d2f7b342.squirrel@manage.unbit.it>
References: <0255399bee51bd1c3df0a091a91f8cf3.squirrel@manage.unbit.it>
	<CAMSv6X1CMwYiT9ouY60MkP2uAzx5Le_d8X4e4ST7pkjn3RDq6g@mail.gmail.com>
	<4e1217cd245fb49f9bb2ade2085b6da1.squirrel@manage.unbit.it>
	<CAEynwxu413dsVWhaodv9VJnekMh=MWQRw0Ft0FZVmdVAac3C6g@mail.gmail.com>
	<93c30f14e987d21708add7a61fad0ca4.squirrel@manage.unbit.it>
	<3a1eb74b7391d714aa93b8b4c5f6f7cf.squirrel@manage.unbit.it>
	<CAK5idxQoCbRq5Rr3Zg9Fz1UfmVqP6L1Vm_w6pZvRc=BfWmKo7A@mail.gmail.com>
	<29bcf1e65852114717cc283402b654a6.squirrel@manage.unbit.it>
	<CAK5idxTtSF0r79utFh0Jx2jNoOAgyj-SngHetFTasfv5mxAzeg@mail.gmail.com>
	<0a196a56fe09d861aa936049b865d908.squirrel@manage.unbit.it>
	<CAK5idxTYTrO0BN0Fo0Th08tYYPHd9mic_RBAA18Z0XWiv_-o2Q@mail.gmail.com>
	<2b7fbb99ef9d417aa8098420959a3ceb.squirrel@manage.unbit.it>
	<df4204f44f9dbb996797e312d2f7b342.squirrel@manage.unbit.it>
Message-ID: <CAMSv6X2drdXOZPv9KRoneYMFUdChjV4QLCMFLr+DU-ZU=b38xg@mail.gmail.com>

Hi Roberto,

On Wed, Apr 4, 2012 at 5:55 PM, Roberto De Ioris <roberto at unbit.it> wrote:
> Ok, i am pretty satisfied with the current code (i have made a pull request).

Trying to figure out what is the status of the pending pull requests,
I came across:

    https://bitbucket.org/pypy/pypy/pull-request/68/pyfile_fromfile-pysys_setargv-and

Should we still try to merge unbit/pypy to pypy/pypy?  It seems that
nowadays the differences are mostly only in cpyext, adding new
functions like Py_Initialize().  Moreover, if the asmgcc failure is
still present, I would just set the options so that "translate
--shared" forces "--gcrootfinder=shadowstack".  Does it seem ok for us
to merge in these conditions?


A bient?t,

Armin.

From roberto at unbit.it  Sat Jun  9 11:34:57 2012
From: roberto at unbit.it (Roberto De Ioris)
Date: Sat, 9 Jun 2012 11:34:57 +0200
Subject: [pypy-dev] update (+patch) on embedding pypy
In-Reply-To: <CAMSv6X2drdXOZPv9KRoneYMFUdChjV4QLCMFLr+DU-ZU=b38xg@mail.gmail.com>
References: <0255399bee51bd1c3df0a091a91f8cf3.squirrel@manage.unbit.it>
	<CAMSv6X1CMwYiT9ouY60MkP2uAzx5Le_d8X4e4ST7pkjn3RDq6g@mail.gmail.com>
	<4e1217cd245fb49f9bb2ade2085b6da1.squirrel@manage.unbit.it>
	<CAEynwxu413dsVWhaodv9VJnekMh=MWQRw0Ft0FZVmdVAac3C6g@mail.gmail.com>
	<93c30f14e987d21708add7a61fad0ca4.squirrel@manage.unbit.it>
	<3a1eb74b7391d714aa93b8b4c5f6f7cf.squirrel@manage.unbit.it>
	<CAK5idxQoCbRq5Rr3Zg9Fz1UfmVqP6L1Vm_w6pZvRc=BfWmKo7A@mail.gmail.com>
	<29bcf1e65852114717cc283402b654a6.squirrel@manage.unbit.it>
	<CAK5idxTtSF0r79utFh0Jx2jNoOAgyj-SngHetFTasfv5mxAzeg@mail.gmail.com>
	<0a196a56fe09d861aa936049b865d908.squirrel@manage.unbit.it>
	<CAK5idxTYTrO0BN0Fo0Th08tYYPHd9mic_RBAA18Z0XWiv_-o2Q@mail.gmail.com>
	<2b7fbb99ef9d417aa8098420959a3ceb.squirrel@manage.unbit.it>
	<df4204f44f9dbb996797e312d2f7b342.squirrel@manage.unbit.it>
	<CAMSv6X2drdXOZPv9KRoneYMFUdChjV4QLCMFLr+DU-ZU=b38xg@mail.gmail.com>
Message-ID: <5cb8f02436c7330a5b550284cf6f9222.squirrel@manage.unbit.it>


> Hi Roberto,
>
> On Wed, Apr 4, 2012 at 5:55 PM, Roberto De Ioris <roberto at unbit.it> wrote:
>> Ok, i am pretty satisfied with the current code (i have made a pull
>> request).
>
> Trying to figure out what is the status of the pending pull requests,
> I came across:
>
>     https://bitbucket.org/pypy/pypy/pull-request/68/pyfile_fromfile-pysys_setargv-and
>
> Should we still try to merge unbit/pypy to pypy/pypy?  It seems that
> nowadays the differences are mostly only in cpyext, adding new
> functions like Py_Initialize().  Moreover, if the asmgcc failure is
> still present, I would just set the options so that "translate
> --shared" forces "--gcrootfinder=shadowstack".  Does it seem ok for us
> to merge in these conditions?
>

Hi Armin, the whole "newer" code is under cpyext. Last time i checked (a
week ago) the asmgcc failure was still there, so i think it is better to
force shadowstack when --shared is specified.


-- 
Roberto De Ioris
http://unbit.it

From arigo at tunes.org  Sat Jun  9 12:29:17 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 9 Jun 2012 12:29:17 +0200
Subject: [pypy-dev] update (+patch) on embedding pypy
In-Reply-To: <5cb8f02436c7330a5b550284cf6f9222.squirrel@manage.unbit.it>
References: <0255399bee51bd1c3df0a091a91f8cf3.squirrel@manage.unbit.it>
	<CAMSv6X1CMwYiT9ouY60MkP2uAzx5Le_d8X4e4ST7pkjn3RDq6g@mail.gmail.com>
	<4e1217cd245fb49f9bb2ade2085b6da1.squirrel@manage.unbit.it>
	<CAEynwxu413dsVWhaodv9VJnekMh=MWQRw0Ft0FZVmdVAac3C6g@mail.gmail.com>
	<93c30f14e987d21708add7a61fad0ca4.squirrel@manage.unbit.it>
	<3a1eb74b7391d714aa93b8b4c5f6f7cf.squirrel@manage.unbit.it>
	<CAK5idxQoCbRq5Rr3Zg9Fz1UfmVqP6L1Vm_w6pZvRc=BfWmKo7A@mail.gmail.com>
	<29bcf1e65852114717cc283402b654a6.squirrel@manage.unbit.it>
	<CAK5idxTtSF0r79utFh0Jx2jNoOAgyj-SngHetFTasfv5mxAzeg@mail.gmail.com>
	<0a196a56fe09d861aa936049b865d908.squirrel@manage.unbit.it>
	<CAK5idxTYTrO0BN0Fo0Th08tYYPHd9mic_RBAA18Z0XWiv_-o2Q@mail.gmail.com>
	<2b7fbb99ef9d417aa8098420959a3ceb.squirrel@manage.unbit.it>
	<df4204f44f9dbb996797e312d2f7b342.squirrel@manage.unbit.it>
	<CAMSv6X2drdXOZPv9KRoneYMFUdChjV4QLCMFLr+DU-ZU=b38xg@mail.gmail.com>
	<5cb8f02436c7330a5b550284cf6f9222.squirrel@manage.unbit.it>
Message-ID: <CAMSv6X0qeuEHAPWydquD935YNa=VHP+_YAyYT5XGsi6r2eR8og@mail.gmail.com>

Hi Roberto,

On Sat, Jun 9, 2012 at 11:34 AM, Roberto De Ioris <roberto at unbit.it> wrote:
> Hi Armin, the whole "newer" code is under cpyext. Last time i checked (a
> week ago) the asmgcc failure was still there, so i think it is better to
> force shadowstack when --shared is specified.

Ok, done.  And I closed the pull request then.

Armin

From matti.picus at gmail.com  Sat Jun  9 21:44:10 2012
From: matti.picus at gmail.com (Matti Picus)
Date: Sat, 09 Jun 2012 22:44:10 +0300
Subject: [pypy-dev] merge of win-ordinal branch
Message-ID: <4FD3A78A.5090705@gmail.com>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120609/89ef834a/attachment.html>

From uwe_f_mayer at yahoo.com  Sun Jun 10 01:36:14 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Sat, 9 Jun 2012 16:36:14 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
Message-ID: <1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>

Just to be sure that we are following up on the right path, I added the hack listed below making the JIT memory allocation shared. Then the os.fork() call works just fine for a simple test program (also included below). Obviously that's not an actual solution as the parent and the child will need separate JIT memory allocations, but it pinpoints the issue. I will issue a report to the Cygwin folks, however a fix may or may not be slow in the coming. How hard would it be for Pypy to use malloc() instead of mmap() for Cygwin?. -- Uwe

*** pypy/pypy/rlib/rmmap.py ? ? Wed May 30 08:11:31 2012
--- pypy/pypy/rlib/rmmap.py ? ? Sat Jun ?9 16:23:45 2012
***************
*** 14,19 ****
--- 14,20 ----
? _MS_WINDOWS = os.name == "nt"
? _LINUX = "linux" in sys.platform
? _64BIT = "64bit" in platform.architecture()[0]
+ _CYGWIN = "cygwin" == sys.platform
??
? class RValueError(Exception):
? ? ? def __init__(self, message):
***************
*** 692,698 ****
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
? ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
? ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
--- 693,703 ----
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? if _CYGWIN:
! ? ? ? ? ? ? # XXX Hack: JIT memory should be private but Cygwin's fork() fails
! ? ? ? ? ? ? flags = MAP_SHARED | MAP_ANONYMOUS
! ? ? ? ? else:
! ? ? ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
? ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
? ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)

PS Here's a little test program.

import sys, os
from test import pystone
# run some code that uses the JIT
print "top-level process:", pystone.main()
print "top-level process:", pystone.main()

pid = os.fork()
if pid == 0:
? ? print "I'm the child, my PID is", os.getpid()
? ? print "child process:", pystone.main()
? ? print "child process:", pystone.main()
? ? sys.exit()
else:
? ? print "I'm the parent, my PID is", os.getpid(), " the child PID is", pid
? ? print "parent process:", pystone.main()
? ? print "parent process:", pystone.main()
? ? sys.exit()




________________________________
 From: Armin Rigo <arigo at tunes.org>
To: Uwe F. Mayer <uwe_f_mayer at yahoo.com> 
Cc: Amaury Forgeot d'Arc <amauryfa at gmail.com>; Matti Picus <matti.picus at gmail.com>; "pypy-dev at python.org" <pypy-dev at python.org> 
Sent: Thursday, June 7, 2012 10:27 PM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin
 
Hi Uwe,

On Thu, Jun 7, 2012 at 9:21 PM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> ? ? ? 3 [main] pypy-c 11788 fixup_mmaps_after_fork: ReadProcessMemory failed
> for MAP_PRIVATE address 0xB0010000, Win32 error 998

Yes, that's one of the mmap addresses used by the JIT, if available.

More to the point, googling for "fork cygwin mmap" shows that there
are open bugs with cygwin's fork() in the presence of MAP_PRIVATE
mmaps.? That's obviously why it fails.

I don't really know what we can reasonably do about that, short of
asking people to use some specific versions of cygwin where it seems
to work.? I can imagine what we could unreasonably do as work-around,
but that requires a lot of work and gross hacks that don't seem
appropriate to put in the pypy source just to support cygwin...? I
would recommend that you try to see it fixed on the cygwin side.


A bient?t,

Armin.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120609/a45a1717/attachment.html>

From amauryfa at gmail.com  Sun Jun 10 11:12:02 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sun, 10 Jun 2012 11:12:02 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
Message-ID: <CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>

2012/6/10 Uwe F. Mayer <uwe_f_mayer at yahoo.com>

> How hard would it be for Pypy to use malloc() instead of mmap() for Cygwin?


I think memory allocated for JIT must be marked with PROT_EXEC (it contains
machine code),
so malloc() is not an option.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120610/71f2e88b/attachment-0001.html>

From alex.pyattaev at gmail.com  Sun Jun 10 17:05:23 2012
From: alex.pyattaev at gmail.com (Alex Pyattaev)
Date: Sun, 10 Jun 2012 18:05:23 +0300
Subject: [pypy-dev] cppyy use case question
In-Reply-To: <alpine.LNX.2.00.1206071525290.1119@pc3-wlav.lbl.gov>
References: <21606049.Qf17BoIqhp@hunter-laptop>
	<1543846.T9Sm08LvAA@hunter-laptop>
	<alpine.LNX.2.00.1206071525290.1119@pc3-wlav.lbl.gov>
Message-ID: <2469357.hbmNeFF51u@hunter-laptop>

Great, that actually works.
torstai 07 kes?kuu 2012 15:34:33 wlavrijsen at lbl.gov kirjoitti:
> Hi Alex,
> 
> > With respect to build, I installed root from gentoo repository, and the
> > problem there is that headers are in /usr/include/root and the actual libs
> > are in /usr/lib/root, but the pypy build expects libs to be in
> > /usr/include/root/lib if you specify SYSROOT=/usr/include/root.
> 
> now picking up include and libdirs via root-config, but then it does require
> that tool to be available through PATH (it should be under $ROOTSYS/bin).
> If not (e.g. standalone Reflex), it will still fall back to
> $ROOTSYS/include etc.
> 
> Best regards,
>             Wim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120610/3ec9f2b7/attachment.html>

From jasper at dqi06.de  Mon Jun 11 00:39:39 2012
From: jasper at dqi06.de (Jasper Schulz)
Date: Mon, 11 Jun 2012 00:39:39 +0200
Subject: [pypy-dev] PyPy-Sprint Leipzig 2012 sign up
Message-ID: <CAPrfWk_x8431ETy5D+jarWoDNwmH=rUJDVCMxtRPq67KGCHt-A@mail.gmail.com>

Hey guys,

my name is Jasper and I'd like to come to Leipzig. I think I'm gonna
do couch-surfing at some friends, so there is no need for any hotel
for me.
I'm not sure when I'll arrive, maybe I can travel with Christian
Tismer, since he lives round the corner as far as I know.

So, I'm really looking forward to meet you and get my brain crushed :-).


Regards
Jasper

From arigo at tunes.org  Mon Jun 11 09:46:37 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Jun 2012 09:46:37 +0200
Subject: [pypy-dev] PyPy-Sprint Leipzig 2012 sign up
In-Reply-To: <CAPrfWk_x8431ETy5D+jarWoDNwmH=rUJDVCMxtRPq67KGCHt-A@mail.gmail.com>
References: <CAPrfWk_x8431ETy5D+jarWoDNwmH=rUJDVCMxtRPq67KGCHt-A@mail.gmail.com>
Message-ID: <CAMSv6X1eBRHd68dwVrai5f7uv_zqyyvXDGpo1HaAOQnLktnrnw@mail.gmail.com>

Hi Jasper,

On Mon, Jun 11, 2012 at 12:39 AM, Jasper Schulz <jasper at dqi06.de> wrote:
> So, I'm really looking forward to meet you and get my brain crushed :-).

Welcome :-)


Armin

From naylor.b.david at gmail.com  Mon Jun 11 12:31:53 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Mon, 11 Jun 2012 12:31:53 +0200
Subject: [pypy-dev] Pypy 1.9 with CLI and JVM backends
Message-ID: <201206111231.56348.naylor.b.david@gmail.com>

Hi,

I'm in the process of preparing the port for FreeBSD of pypy-1.9.  The patches 
required to run under FreeBSD are minimal (1 submittable patch, 1 specific 
patch not applicable for pypy).  Thanks!  

I was just wondering on the state of CLI and JVM backends in pypy-1.9.  Does 
translating work for either of those backends?  

I failed to translate pypy using the CLI backend with error "TypeError: Bad 
overloading".  I recall that was perhaps a 64bit vs 32bit error and that, 
maybe, translating for CLI will work in a 32bit environment?  

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120611/f808cd6e/attachment.pgp>

From fijall at gmail.com  Mon Jun 11 12:39:45 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 11 Jun 2012 12:39:45 +0200
Subject: [pypy-dev] Pypy 1.9 with CLI and JVM backends
In-Reply-To: <201206111231.56348.naylor.b.david@gmail.com>
References: <201206111231.56348.naylor.b.david@gmail.com>
Message-ID: <CAK5idxRa6G+0o2xijjeZ1ZLSnMP43d7pP63=avMFWM=n+mxapA@mail.gmail.com>

On Mon, Jun 11, 2012 at 12:31 PM, David Naylor <naylor.b.david at gmail.com>wrote:

> Hi,
>
> I'm in the process of preparing the port for FreeBSD of pypy-1.9.  The
> patches
> required to run under FreeBSD are minimal (1 submittable patch, 1 specific
> patch not applicable for pypy).  Thanks!
>
> I was just wondering on the state of CLI and JVM backends in pypy-1.9.
>  Does
> translating work for either of those backends?
>
> I failed to translate pypy using the CLI backend with error "TypeError: Bad
> overloading".  I recall that was perhaps a 64bit vs 32bit error and that,
> maybe, translating for CLI will work in a 32bit environment?
>
> Regards
>

It's in a might-be-working-but-not-really useful state. I don't think it
ever worked ion 64 bit anyway.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120611/bc2cd9ed/attachment.html>

From michal at bendowski.pl  Mon Jun 11 12:59:55 2012
From: michal at bendowski.pl (Michal Bendowski)
Date: Mon, 11 Jun 2012 12:59:55 +0200
Subject: [pypy-dev] Pypy 1.9 with CLI and JVM backends
In-Reply-To: <201206111231.56348.naylor.b.david@gmail.com>
References: <201206111231.56348.naylor.b.david@gmail.com>
Message-ID: <CAFKZzE-vH_HOe7HYVCbh7AokkvwRAzN1eDFt4xM5QzsqEfL0xg@mail.gmail.com>

The JVM backend worked in February ;) I will be looking into making it
translate the interpreter again soon. For now it requires a 32bit
environment.

Micha?

On Mon, Jun 11, 2012 at 12:31 PM, David Naylor <naylor.b.david at gmail.com>wrote:

> Hi,
>
> I'm in the process of preparing the port for FreeBSD of pypy-1.9.  The
> patches
> required to run under FreeBSD are minimal (1 submittable patch, 1 specific
> patch not applicable for pypy).  Thanks!
>
> I was just wondering on the state of CLI and JVM backends in pypy-1.9.
>  Does
> translating work for either of those backends?
>
> I failed to translate pypy using the CLI backend with error "TypeError: Bad
> overloading".  I recall that was perhaps a 64bit vs 32bit error and that,
> maybe, translating for CLI will work in a 32bit environment?
>
> Regards
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120611/66e6a2c1/attachment.html>

From arigo at tunes.org  Mon Jun 11 13:59:55 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 11 Jun 2012 13:59:55 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
Message-ID: <CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>

Hi,

On Sun, Jun 10, 2012 at 11:12 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> I think memory allocated for JIT must be marked with PROT_EXEC (it contains
> machine code),
> so malloc() is not an option.

Exactly.  On Linux it would work, because you can use mprotect() to
make even malloc()ed memory executable, but I doubt this non-Posix
extension works on cygwin.  What might work instead would be using the
Windows API instead of the POSIX one.  In pypy/rlib/rmmap.py, look for
"def alloc" and "def free": there are the two versions.


A bient?t,

Armin.

From uwe_f_mayer at yahoo.com  Mon Jun 11 19:27:40 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Mon, 11 Jun 2012 10:27:40 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
Message-ID: <1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>

> ----- Original Message -----
> From: Armin Rigo <arigo at tunes.org>
> Sent: Monday, June 11, 2012 4:59 AM
> >>On Sun, Jun 10, 2012 at 11:12 AM, Amaury Forgeot d'Arc?<amauryfa at gmail.com> wrote:
> >> I think memory allocated for JIT must be marked with PROT_EXEC (it contains
> >> machine code),?so malloc() is not an option.
>?Exactly.? On Linux it would work, because you can use mprotect() to
> make even malloc()ed memory executable, but I doubt this non-Posix
> extension works on cygwin.? What might work instead would be using the
> Windows API instead of the POSIX one.? In pypy/rlib/rmmap.py, look for
> "def alloc" and "def free": there are the two versions.


Cygwin implements the POSIX API on top of the Windows API, and some things are tricky to implement, and memory handling and permissions tend to be part of those. To see if it may just work, I modified rmmap.pyto use malloc() and free(), and indeed pypy seems to work just fine. Following up on this email I also added an mprotect() call, which does exist under Cygwin. However,?POSIX says that the behavior of mprotect() is unspecified?if it is applied to a region of memory that was not obtained?via mmap(2), and indeed?this fails.

Here's the patch that works, using malloc() and free(), and not using mprotect().

*** pypy/pypy/rlib/rmmap.pyMon Jun 11 10:18:55 2012
--- pypy/pypy/rlib/rmmap.pyMon Jun 11 10:16:12 2012
***************
*** 14,19 ****
--- 14,20 ----
? _MS_WINDOWS = os.name == "nt"
? _LINUX = "linux" in sys.platform
? _64BIT = "64bit" in platform.architecture()[0]
+ _CYGWIN = "cygwin" == sys.platform
??
? class RValueError(Exception):
? ? ? def __init__(self, message):
***************
*** 115,120 ****
--- 116,126 ----
??
? PTR = rffi.CCHARP
??
+ if _CYGWIN:
+ ? ? c_malloc, _ = external('malloc', [size_t], PTR)
+ ? ? #c_mprotect, _ = external('mprotect', [PTR, size_t, rffi.INT], rffi.INT)
+ ? ? c_free, _ = external('free', [PTR], lltype.Void)
+?
? c_memmove, _ = external('memmove', [PTR, PTR, size_t], lltype.Void)
??
? if _POSIX:
***************
*** 692,714 ****
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
! ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
! ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
! ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? # some systems (some versions of OS/X?) complain if they
! ? ? ? ? ? ? # are passed a non-zero address. ?Try again.
! ? ? ? ? ? ? hintp = rffi.cast(PTR, 0)
! ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
? ? ? ? ? ? ? ? ? raise MemoryError
? ? ? ? ? else:
! ? ? ? ? ? ? hint.pos += map_size
? ? ? ? ? return res
? ? ? alloc._annenforceargs_ = (int,)
??
! ? ? free = c_munmap_safe
??
? elif _MS_WINDOWS:
? ? ? def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):
--- 698,739 ----
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? if _CYGWIN:
! ? ? ? ? ? ? # XXX: JIT memory should be using mmap MAP_PRIVATE with
! ? ? ? ? ? ? # ? ? ?PROT_EXEC but Cygwin's fork() fails
! ? ? ? ? ? ? res = c_malloc(map_size)
! ? ? ? ? ? ? if res == rffi.cast(PTR, 0):
? ? ? ? ? ? ? ? ? raise MemoryError
+ ? ? ? ? ? ? # XXX POSIX says that the behavior of mprotect() is unspecified
+ ? ? ? ? ? ? # if it is applied to a region of memory that was not obtained
+ ? ? ? ? ? ? # via mmap(2), and indeed it does not work here.
+ ? ? ? ? ? ? # prot = PROT_EXEC | PROT_READ | PROT_WRITE
+ ? ? ? ? ? ? # res2 = c_mprotect(res, map_size, prot)
+ ? ? ? ? ? ? # if res2 == -1:
+ ? ? ? ? ? ? # ? ? errno = rposix.get_errno()
+ ? ? ? ? ? ? # ? ? raise OSError(errno, os.strerror(errno))
? ? ? ? ? else:
! ? ? ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
! ? ? ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
! ? ? ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
! ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? ? ? # some systems (some versions of OS/X?) complain if they
! ? ? ? ? ? ? ? ? # are passed a non-zero address. ?Try again.
! ? ? ? ? ? ? ? ? hintp = rffi.cast(PTR, 0)
! ? ? ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? ? ? ? ? raise MemoryError
! ? ? ? ? ? ? else:
! ? ? ? ? ? ? ? ? hint.pos += map_size
? ? ? ? ? return res
+?
? ? ? alloc._annenforceargs_ = (int,)
??
! ? ? if _CYGWIN:
! ? ? ? ? free = c_free
! ? ? else:
! ? ? ? ? free = c_munmap_safe
??
? elif _MS_WINDOWS:
? ? ? def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):

From csparr at ucsd.edu  Tue Jun 12 00:50:08 2012
From: csparr at ucsd.edu (Cameron Sparr)
Date: Mon, 11 Jun 2012 16:50:08 -0600
Subject: [pypy-dev] problems moving to PyPy 1.9 and GLIBC_2.7
Message-ID: <4FD67620.6010109@ucsd.edu>

Hi all,

I'm trying to run PyPy 1.9 (Was running 1.8 just fine) but get the message:

      /lib64/libc.so.6: version `GLIBC_2.7' not found (required by ./pypy)

I tried downloading and building lib C version 2.7 but this causes a 
host of other problems on my system. Was moving to Lib C 2.7 just a 
difference in the development machine being used to build the PyPy 
binary? Any possibility of finding a PyPy 1.9 version that does not 
require GLIBC_2.7 (not sure what exact version was required by PyPy 1.8)

Best,
Cameron


-- 
Cameron Sparr
SIParCS Intern
NCAR \ CISL \ User Services Section (USS)
phone:  303-497-2443
mobile: 503-327-5964
email:  csparr at ucar.edu


From fijall at gmail.com  Tue Jun 12 09:39:09 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 12 Jun 2012 09:39:09 +0200
Subject: [pypy-dev] problems moving to PyPy 1.9 and GLIBC_2.7
In-Reply-To: <4FD67620.6010109@ucsd.edu>
References: <4FD67620.6010109@ucsd.edu>
Message-ID: <CAK5idxTNoOCrd5=NdRTCsmuLwAhMMKGeGFwtVuz4KzfkETa9aQ@mail.gmail.com>

This is a general problem of binary distributions on linux - they are just
known not to work exceptionally well.

I guess the best options is really to build pypy yourself (although there
might exist a solution to this particular problem that I'm not aware of,
I've never seen it before).

Answering your question, the build system was not modified according to my
knowledge.

On Tue, Jun 12, 2012 at 12:50 AM, Cameron Sparr <csparr at ucsd.edu> wrote:

> Hi all,
>
> I'm trying to run PyPy 1.9 (Was running 1.8 just fine) but get the message:
>
>     /lib64/libc.so.6: version `GLIBC_2.7' not found (required by ./pypy)
>
> I tried downloading and building lib C version 2.7 but this causes a host
> of other problems on my system. Was moving to Lib C 2.7 just a difference
> in the development machine being used to build the PyPy binary? Any
> possibility of finding a PyPy 1.9 version that does not require GLIBC_2.7
> (not sure what exact version was required by PyPy 1.8)
>
> Best,
> Cameron
>
>
> --
> Cameron Sparr
> SIParCS Intern
> NCAR \ CISL \ User Services Section (USS)
> phone:  303-497-2443
> mobile: 503-327-5964
> email:  csparr at ucar.edu
>
> ______________________________**_________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120612/3e369fe4/attachment.html>

From info at mail.cymbae3.com  Tue Jun 12 09:57:53 2012
From: info at mail.cymbae3.com (VEDI)
Date: Tue, 12 Jun 2012 09:57:53 +0200
Subject: [pypy-dev] Ground Board Display: Neu Outdoor-Display bei Vedi
Message-ID: <8f8713577f58f6cfc69c8a8f2ee040d1@mail.cymbae3.com>





 


Outdoor-Display doppelseitig f?r Ihre externe Kommunikation 
Eins?tze zur Verankerung im Boden (Rasen) 

Gummiseile f?r eine sichere Befestigung am Display 
Einfache Montage: 1 Person in 10 Minuten 
Transporttasche inklusive   
* Bild separat erh?ltlich 



     Bestellen Sie Online in nur 5 Minuten 

 
BILITE INDOOR/ BILITE OUTDOOR
Selbsttragende Au?envorrichtung zum einspannen einer Plane. Aluminiumkonstruktion. 14 Modelle verf?gbar. Benutzung der Vorder oder Vorder und R?ckseite. Aufbauzeit in1 Stunde mit einer Person im Inneren und 2 Personnen im Aussenbereich. 
 
PLAKATAU?EN- 
ST?NDER 

Mehrere Modelle verf?gbar. 
Ballastierung durch das F?llen f?r einen gr??eren Windwiderstand. 2 Rollen erleichtern die Beweglichkeit der Vorrichtung. 

 
PLAKATST?NDER 

Erh?ltlich im 
Format A2 + A1. Kommunikation duch Vorder und R?ckseite.  Leichtes Installieren. Stabilisiert durch zwei Scharnieren. 


                           
VEDI express ? 2002-2012 
RollUps Displays, Planen, Messest?nde Pop Ups zu besten Preisen und 
schnellen Lieferzeiten. 
Powered by VEDI  
  Keine Newsletter mehr erhalten 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120612/1d4a2bd4/attachment-0001.html>

From naylor.b.david at gmail.com  Tue Jun 12 11:58:05 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Tue, 12 Jun 2012 11:58:05 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
Message-ID: <201206121158.11984.naylor.b.david@gmail.com>

Hi,

FYI, translating pypy for the CLI backend using mono-2.11.1 in a 32bit environment results in the following error:
[Timer] Timings:
[Timer] annotate                       --- 309.3 s
[Timer] rtype_ootype                   --- 326.7 s
[Timer] ==========================================
[Timer] Total:                         --- 636.0 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 308, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed
[translation:ERROR]     return self._execute(goals, task_skip = self._maybe_skip())
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 383, in task_rtype_ootype
[translation:ERROR]     crash_on_first_typeerror=insist)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 211, in specialize
[translation:ERROR]     self.specialize_more_blocks()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 254, in specialize_more_blocks
[translation:ERROR]     self.specialize_block(block)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 408, in specialize_block
[translation:ERROR]     self.gottypererror(e, block, hop.spaceop, newops)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 406, in specialize_block
[translation:ERROR]     self.translate_hl_to_ll(hop, varmapping)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 535, in translate_hl_to_ll
[translation:ERROR]     resultvar = hop.dispatch()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 768, in dispatch
[translation:ERROR]     return translate_meth(self)
[translation:ERROR]    File "<18375-codegen /tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py:610>", line 4, in 
translate_op_simple_call
[translation:ERROR]     return r_arg1.rtype_simple_call(hop)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 126, in rtype_simple_call
[translation:ERROR]     return self._call(hop2)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rbuiltin.py", line 117, in _call
[translation:ERROR]     v_result = bltintyper(hop2, **kwds_i)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rlib/jit.py", line 802, in specialize_call
[translation:ERROR]     v_value = hop.inputarg(repr, arg=2)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 788, in inputarg
[translation:ERROR]     return self.llops.convertvar(v, r_binding, converted_to)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/rtyper.py", line 933, in convertvar
[translation:ERROR]     (r_from, r_to))
[translation:ERROR]  TyperError: don't know how to convert from <StringRepr String> to <StringRepr * GcStruct rpy_string { hash, chars }>
[translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
[translation:ERROR] .. block at 12 with 1 exits
[translation:ERROR] .. v552 = simple_call((function _set_param), driver_0, name_0, value_0)

I know the CLI backend is not really supported but I am sharing this in the event someone is interested.  

Regards,

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120612/2d9991f9/attachment.pgp>

From naylor.b.david at gmail.com  Tue Jun 12 12:11:54 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Tue, 12 Jun 2012 12:11:54 +0200
Subject: [pypy-dev] Pypy 1.9 with CLI and JVM backends
In-Reply-To: <CAFKZzE-vH_HOe7HYVCbh7AokkvwRAzN1eDFt4xM5QzsqEfL0xg@mail.gmail.com>
References: <201206111231.56348.naylor.b.david@gmail.com>
	<CAFKZzE-vH_HOe7HYVCbh7AokkvwRAzN1eDFt4xM5QzsqEfL0xg@mail.gmail.com>
Message-ID: <201206121212.01033.naylor.b.david@gmail.com>

On Monday, 11 June 2012 12:59:55 Michal Bendowski wrote:
> The JVM backend worked in February ;) I will be looking into making it
> translate the interpreter again soon. For now it requires a 32bit
> environment.

FYI, here is the error I get when translating in a 32 environment:
[Timer] Timings:
[Timer] annotate                       --- 285.0 s
[Timer] rtype_ootype                   --- 362.7 s
[Timer] backendopt_ootype              ---  53.8 s
[Timer] source_jvm                     ---  17.5 s
[Timer] ==========================================
[Timer] Total:                         --- 718.9 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 308, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed
[translation:ERROR]     return self._execute(goals, task_skip = self._maybe_skip())
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 703, in task_source_jvm
[translation:ERROR]     self.jvmsource = self.gen.generate_source()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/jvm/genjvm.py", line 266, in generate_source
[translation:ERROR]     GenOO.generate_source(self)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 68, in generate_source
[translation:ERROR]     self.gen_pendings()
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 83, in gen_pendings
[translation:ERROR]     node.render(self.ilasm)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/jvm/node.py", line 902, in render
[translation:ERROR]     method.render(gen)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 129, in render
[translation:ERROR]     self.render_exc_handling_block(block)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 156, in 
render_exc_handling_block
[translation:ERROR]     self._render_op(op)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/jvm/node.py", line 577, in _render_op
[translation:ERROR]     OOFunction._render_op(self, op)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 377, in _render_op
[translation:ERROR]     instr_list.render(self.generator, op)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/metavm.py", line 246, in render
[translation:ERROR]     instr.render(generator, op)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/metavm.py", line 456, in render
[translation:ERROR]     is_primitive = get_primitive_name(callee)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/metavm.py", line 450, in 
get_primitive_name
[translation:ERROR]     return sm._name.rsplit('.', 1)
[translation:ERROR]    File "/tmp/usr/ports/lang/pypy/work/pypy-pypy-341e1e3821ff/pypy/rpython/lltypesystem/lltype.py", line 1171, in __getattr__
[translation:ERROR]     field_name))
[translation:ERROR]  AttributeError: <Func (  ) -> Signed> instance has no field '_name'

I haven't received any requests for use of the JVM backend on FreeBSD so this is not urgent.  

Thanks, and good luck.  
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120612/a9ba8af3/attachment.pgp>

From uwe_f_mayer at yahoo.com  Wed Jun 13 03:50:11 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Tue, 12 Jun 2012 18:50:11 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
Message-ID: <1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>

I have made a final patch (as of now) and linked it from:
http://www.tux.org/~mayer/cygwin/pypy/index.html

See the bottom third of the page, the patch against the trunk of the mercurial repository is
http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch

With this pypy builds under Cygwin, and os.fork() works.
It would be good if this could get merged into the trunk.
--Uwe


----- Original Message -----
From: Uwe F. Mayer <uwe_f_mayer at yahoo.com>
To: Armin Rigo <arigo at tunes.org>; Amaury Forgeot d'Arc <amauryfa at gmail.com>
Cc: Matti Picus <matti.picus at gmail.com>; "pypy-dev at python.org" <pypy-dev at python.org>
Sent: Monday, June 11, 2012 10:27 AM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin

> ----- Original Message -----
> From: Armin Rigo <arigo at tunes.org>
> Sent: Monday, June 11, 2012 4:59 AM
> >>On Sun, Jun 10, 2012 at 11:12 AM, Amaury Forgeot d'Arc?<amauryfa at gmail.com> wrote:
> >> I think memory allocated for JIT must be marked with PROT_EXEC (it contains
> >> machine code),?so malloc() is not an option.
>?Exactly.? On Linux it would work, because you can use mprotect() to
> make even malloc()ed memory executable, but I doubt this non-Posix
> extension works on cygwin.? What might work instead would be using the
> Windows API instead of the POSIX one.? In pypy/rlib/rmmap.py, look for
> "def alloc" and "def free": there are the two versions.


Cygwin implements the POSIX API on top of the Windows API, and some things are tricky to implement, and memory handling and permissions tend to be part of those. To see if it may just work, I modified rmmap.pyto use malloc() and free(), and indeed pypy seems to work just fine. Following up on this email I also added an mprotect() call, which does exist under Cygwin. However,?POSIX says that the behavior of mprotect() is unspecified?if it is applied to a region of memory that was not obtained?via mmap(2), and indeed?this fails.

Here's the patch that works, using malloc() and free(), and not using mprotect().

*** pypy/pypy/rlib/rmmap.pyMon Jun 11 10:18:55 2012
--- pypy/pypy/rlib/rmmap.pyMon Jun 11 10:16:12 2012
***************
*** 14,19 ****
--- 14,20 ----
? _MS_WINDOWS = os.name == "nt"
? _LINUX = "linux" in sys.platform
? _64BIT = "64bit" in platform.architecture()[0]
+ _CYGWIN = "cygwin" == sys.platform
??
? class RValueError(Exception):
? ? ? def __init__(self, message):
***************
*** 115,120 ****
--- 116,126 ----
??
? PTR = rffi.CCHARP
??
+ if _CYGWIN:
+ ? ? c_malloc, _ = external('malloc', [size_t], PTR)
+ ? ? #c_mprotect, _ = external('mprotect', [PTR, size_t, rffi.INT], rffi.INT)
+ ? ? c_free, _ = external('free', [PTR], lltype.Void)
+?
? c_memmove, _ = external('memmove', [PTR, PTR, size_t], lltype.Void)
??
? if _POSIX:
***************
*** 692,714 ****
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
! ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
! ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
! ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? # some systems (some versions of OS/X?) complain if they
! ? ? ? ? ? ? # are passed a non-zero address. ?Try again.
! ? ? ? ? ? ? hintp = rffi.cast(PTR, 0)
! ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
? ? ? ? ? ? ? ? ? raise MemoryError
? ? ? ? ? else:
! ? ? ? ? ? ? hint.pos += map_size
? ? ? ? ? return res
? ? ? alloc._annenforceargs_ = (int,)
??
! ? ? free = c_munmap_safe
??
? elif _MS_WINDOWS:
? ? ? def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):
--- 698,739 ----
? ? ? ? ? so the memory has the executable bit set and gets allocated
? ? ? ? ? internally in case of a sandboxed process.
? ? ? ? ? """
! ? ? ? ? if _CYGWIN:
! ? ? ? ? ? ? # XXX: JIT memory should be using mmap MAP_PRIVATE with
! ? ? ? ? ? ? # ? ? ?PROT_EXEC but Cygwin's fork() fails
! ? ? ? ? ? ? res = c_malloc(map_size)
! ? ? ? ? ? ? if res == rffi.cast(PTR, 0):
? ? ? ? ? ? ? ? ? raise MemoryError
+ ? ? ? ? ? ? # XXX POSIX says that the behavior of mprotect() is unspecified
+ ? ? ? ? ? ? # if it is applied to a region of memory that was not obtained
+ ? ? ? ? ? ? # via mmap(2), and indeed it does not work here.
+ ? ? ? ? ? ? # prot = PROT_EXEC | PROT_READ | PROT_WRITE
+ ? ? ? ? ? ? # res2 = c_mprotect(res, map_size, prot)
+ ? ? ? ? ? ? # if res2 == -1:
+ ? ? ? ? ? ? # ? ? errno = rposix.get_errno()
+ ? ? ? ? ? ? # ? ? raise OSError(errno, os.strerror(errno))
? ? ? ? ? else:
! ? ? ? ? ? ? flags = MAP_PRIVATE | MAP_ANONYMOUS
! ? ? ? ? ? ? prot = PROT_EXEC | PROT_READ | PROT_WRITE
! ? ? ? ? ? ? hintp = rffi.cast(PTR, hint.pos)
! ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? ? ? # some systems (some versions of OS/X?) complain if they
! ? ? ? ? ? ? ? ? # are passed a non-zero address. ?Try again.
! ? ? ? ? ? ? ? ? hintp = rffi.cast(PTR, 0)
! ? ? ? ? ? ? ? ? res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
! ? ? ? ? ? ? ? ? if res == rffi.cast(PTR, -1):
! ? ? ? ? ? ? ? ? ? ? raise MemoryError
! ? ? ? ? ? ? else:
! ? ? ? ? ? ? ? ? hint.pos += map_size
? ? ? ? ? return res
+?
? ? ? alloc._annenforceargs_ = (int,)
??
! ? ? if _CYGWIN:
! ? ? ? ? free = c_free
! ? ? else:
! ? ? ? ? free = c_munmap_safe
??
? elif _MS_WINDOWS:
? ? ? def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):


From Ronny.Pfannschmidt at gmx.de  Wed Jun 13 09:54:06 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 13 Jun 2012 09:54:06 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
Message-ID: <4FD8471E.4040405@gmx.de>

We would appreciate a unified diff or
something that can easily be feed to hg import

the most appreciated way to give us patches is
to have native hg created patches
that we can just feed into hg at our side

(native hg created patches also contain stuff like your name for correct 
attribution)

-- Ronny


On 06/13/2012 03:50 AM, Uwe F. Mayer wrote:
> I have made a final patch (as of now) and linked it from:
> http://www.tux.org/~mayer/cygwin/pypy/index.html
>
> See the bottom third of the page, the patch against the trunk of the mercurial repository is
> http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch
>
> With this pypy builds under Cygwin, and os.fork() works.
> It would be good if this could get merged into the trunk.
> --Uwe
>
>
> ----- Original Message -----
> From: Uwe F. Mayer<uwe_f_mayer at yahoo.com>
> To: Armin Rigo<arigo at tunes.org>; Amaury Forgeot d'Arc<amauryfa at gmail.com>
> Cc: Matti Picus<matti.picus at gmail.com>; "pypy-dev at python.org"<pypy-dev at python.org>
> Sent: Monday, June 11, 2012 10:27 AM
> Subject: Re: [pypy-dev] Patches for Pypy under cygwin
>
>> ----- Original Message -----
>> From: Armin Rigo<arigo at tunes.org>
>> Sent: Monday, June 11, 2012 4:59 AM
>>>> On Sun, Jun 10, 2012 at 11:12 AM, Amaury Forgeot d'Arc<amauryfa at gmail.com>  wrote:
>>>> I think memory allocated for JIT must be marked with PROT_EXEC (it contains
>>>> machine code), so malloc() is not an option.
>>   Exactly.  On Linux it would work, because you can use mprotect() to
>> make even malloc()ed memory executable, but I doubt this non-Posix
>> extension works on cygwin.  What might work instead would be using the
>> Windows API instead of the POSIX one.  In pypy/rlib/rmmap.py, look for
>> "def alloc" and "def free": there are the two versions.
>
>
> Cygwin implements the POSIX API on top of the Windows API, and some things are tricky to implement, and memory handling and permissions tend to be part of those. To see if it may just work, I modified rmmap.pyto use malloc() and free(), and indeed pypy seems to work just fine. Following up on this email I also added an mprotect() call, which does exist under Cygwin. However, POSIX says that the behavior of mprotect() is unspecified if it is applied to a region of memory that was not obtained via mmap(2), and indeed this fails.
>
> Here's the patch that works, using malloc() and free(), and not using mprotect().
>
> *** pypy/pypy/rlib/rmmap.pyMon Jun 11 10:18:55 2012
> --- pypy/pypy/rlib/rmmap.pyMon Jun 11 10:16:12 2012
> ***************
> *** 14,19 ****
> --- 14,20 ----
>    _MS_WINDOWS = os.name == "nt"
>    _LINUX = "linux" in sys.platform
>    _64BIT = "64bit" in platform.architecture()[0]
> + _CYGWIN = "cygwin" == sys.platform
>
>    class RValueError(Exception):
>        def __init__(self, message):
> ***************
> *** 115,120 ****
> --- 116,126 ----
>
>    PTR = rffi.CCHARP
>
> + if _CYGWIN:
> +     c_malloc, _ = external('malloc', [size_t], PTR)
> +     #c_mprotect, _ = external('mprotect', [PTR, size_t, rffi.INT], rffi.INT)
> +     c_free, _ = external('free', [PTR], lltype.Void)
> +
>    c_memmove, _ = external('memmove', [PTR, PTR, size_t], lltype.Void)
>
>    if _POSIX:
> ***************
> *** 692,714 ****
>            so the memory has the executable bit set and gets allocated
>            internally in case of a sandboxed process.
>            """
> !         flags = MAP_PRIVATE | MAP_ANONYMOUS
> !         prot = PROT_EXEC | PROT_READ | PROT_WRITE
> !         hintp = rffi.cast(PTR, hint.pos)
> !         res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
> !         if res == rffi.cast(PTR, -1):
> !             # some systems (some versions of OS/X?) complain if they
> !             # are passed a non-zero address.  Try again.
> !             hintp = rffi.cast(PTR, 0)
> !             res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
> !             if res == rffi.cast(PTR, -1):
>                    raise MemoryError
>            else:
> !             hint.pos += map_size
>            return res
>        alloc._annenforceargs_ = (int,)
>
> !     free = c_munmap_safe
>
>    elif _MS_WINDOWS:
>        def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):
> --- 698,739 ----
>            so the memory has the executable bit set and gets allocated
>            internally in case of a sandboxed process.
>            """
> !         if _CYGWIN:
> !             # XXX: JIT memory should be using mmap MAP_PRIVATE with
> !             #      PROT_EXEC but Cygwin's fork() fails
> !             res = c_malloc(map_size)
> !             if res == rffi.cast(PTR, 0):
>                    raise MemoryError
> +             # XXX POSIX says that the behavior of mprotect() is unspecified
> +             # if it is applied to a region of memory that was not obtained
> +             # via mmap(2), and indeed it does not work here.
> +             # prot = PROT_EXEC | PROT_READ | PROT_WRITE
> +             # res2 = c_mprotect(res, map_size, prot)
> +             # if res2 == -1:
> +             #     errno = rposix.get_errno()
> +             #     raise OSError(errno, os.strerror(errno))
>            else:
> !             flags = MAP_PRIVATE | MAP_ANONYMOUS
> !             prot = PROT_EXEC | PROT_READ | PROT_WRITE
> !             hintp = rffi.cast(PTR, hint.pos)
> !             res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
> !             if res == rffi.cast(PTR, -1):
> !                 # some systems (some versions of OS/X?) complain if they
> !                 # are passed a non-zero address.  Try again.
> !                 hintp = rffi.cast(PTR, 0)
> !                 res = c_mmap_safe(hintp, map_size, prot, flags, -1, 0)
> !                 if res == rffi.cast(PTR, -1):
> !                     raise MemoryError
> !             else:
> !                 hint.pos += map_size
>            return res
> +
>        alloc._annenforceargs_ = (int,)
>
> !     if _CYGWIN:
> !         free = c_free
> !     else:
> !         free = c_munmap_safe
>
>    elif _MS_WINDOWS:
>        def mmap(fileno, length, tagname="", access=_ACCESS_DEFAULT, offset=0):
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From tobias.oberstein at tavendo.de  Wed Jun 13 09:59:26 2012
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Wed, 13 Jun 2012 00:59:26 -0700
Subject: [pypy-dev] 300 bucks reward for fixing PyPy/OpenSSL issue
Message-ID: <634914A010D0B943A035D226786325D43377FA40B3@EXVMBX020-12.exch020.serverdata.net>

Hello all,

I am offering a 300 $ reward to anyone fixing

https://bugs.pypy.org/issue1162

We want to use Twisted on PyPy with OpenSSL on a network server involving threads, and above issue is blocking this.

Anyone?

Cheers,
Tobias

From amauryfa at gmail.com  Wed Jun 13 10:25:33 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 13 Jun 2012 10:25:33 +0200
Subject: [pypy-dev] 300 bucks reward for fixing PyPy/OpenSSL issue
In-Reply-To: <634914A010D0B943A035D226786325D43377FA40B3@EXVMBX020-12.exch020.serverdata.net>
References: <634914A010D0B943A035D226786325D43377FA40B3@EXVMBX020-12.exch020.serverdata.net>
Message-ID: <CAGmFidbht=gkYFLfDdwzEkCTfYWyiLRL8YK7vC7VgRj62ryy6Q@mail.gmail.com>

2012/6/13 Tobias Oberstein <tobias.oberstein at tavendo.de>

> I am offering a 300 $ reward to anyone fixing
>
> https://bugs.pypy.org/issue1162
>
> We want to use Twisted on PyPy with OpenSSL on a network server involving
> threads, and above issue is blocking this.
>
> Anyone?
>

Did you try with a recent nightly build? One created after this change:
http://mail.python.org/pipermail/pypy-commit/2012-June/063032.html

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120613/0857705b/attachment.html>

From tobias.oberstein at tavendo.de  Wed Jun 13 10:48:54 2012
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Wed, 13 Jun 2012 01:48:54 -0700
Subject: [pypy-dev] 300 bucks reward for fixing PyPy/OpenSSL issue
In-Reply-To: <CAGmFidbht=gkYFLfDdwzEkCTfYWyiLRL8YK7vC7VgRj62ryy6Q@mail.gmail.com>
References: <634914A010D0B943A035D226786325D43377FA40B3@EXVMBX020-12.exch020.serverdata.net>
	<CAGmFidbht=gkYFLfDdwzEkCTfYWyiLRL8YK7vC7VgRj62ryy6Q@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D43377FA40B6@EXVMBX020-12.exch020.serverdata.net>

> Did you try with a recent nightly build? One created after this change:
> http://mail.python.org/pipermail/pypy-commit/2012-June/063032.html?

No, I didn't .. I was using a build with last commit from 9th June .. I will try with this commit. Thx!


From arigo at tunes.org  Wed Jun 13 13:06:14 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 13 Jun 2012 13:06:14 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <201206121158.11984.naylor.b.david@gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
Message-ID: <CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>

Hi David,

On Tue, Jun 12, 2012 at 11:58 AM, David Naylor <naylor.b.david at gmail.com> wrote:
> [translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts

Nowadays, you cannot use the JIT with the CLI backend.


Armin

From fijall at gmail.com  Wed Jun 13 13:17:22 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 13 Jun 2012 13:17:22 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
Message-ID: <CAK5idxQ1iGWnPF1pYeMWKEHkuG1NgWKX_j1YG=Z6NpewTD3fJw@mail.gmail.com>

On Wed, Jun 13, 2012 at 1:06 PM, Armin Rigo <arigo at tunes.org> wrote:

> Hi David,
>
> On Tue, Jun 12, 2012 at 11:58 AM, David Naylor <naylor.b.david at gmail.com>
> wrote:
> > [translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
>
> Nowadays, you cannot use the JIT with the CLI backend.
>
>
> Armin
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

That probably does not mean that someone somewhere is not calling
set_param, like from command line parsing.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120613/4de46544/attachment.html>

From arigo at tunes.org  Wed Jun 13 13:20:05 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 13 Jun 2012 13:20:05 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <4FD8471E.4040405@gmx.de>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
	<4FD8471E.4040405@gmx.de>
Message-ID: <CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>

Hi Ronny,

On Wed, Jun 13, 2012 at 9:54 AM, Ronny Pfannschmidt
<Ronny.Pfannschmidt at gmx.de> wrote:
> We would appreciate a unified diff or
> something that can easily be feed to hg import

I don't think I would reject such a patch from these grounds; at least
it *is* a patch, easily usable with "patch", and not, say, entire
files that have been modified from some unknown revision, as we got
too.

I have checked in a slightly edited version of
http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch (hg
revision c7dff5469611).  Please tell me if I missed something.


A bient?t,

Armin.

From arigo at tunes.org  Wed Jun 13 13:24:55 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 13 Jun 2012 13:24:55 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAK5idxQ1iGWnPF1pYeMWKEHkuG1NgWKX_j1YG=Z6NpewTD3fJw@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
	<CAK5idxQ1iGWnPF1pYeMWKEHkuG1NgWKX_j1YG=Z6NpewTD3fJw@mail.gmail.com>
Message-ID: <CAMSv6X2t21n7792OMYH7e8knShcqxCQZU0-QUdWhC6h9JpK2VA@mail.gmail.com>

Hi Fijal,

On Wed, Jun 13, 2012 at 1:17 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> That probably does not mean that someone somewhere is not calling set_param,
> like from command line parsing.

It does mean it.  At RPython level the only callers of
rlib.jit.set_param() are all from module/pypyjit.  For the
command-line parsing, this is done in app_main.py, which will just
fail to import "pypyjit" on a no-jit pypy (and give a nice
explanation).


A bient?t,

Armin.

From naylor.b.david at gmail.com  Wed Jun 13 14:49:06 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Wed, 13 Jun 2012 14:49:06 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
Message-ID: <201206131449.11403.naylor.b.david@gmail.com>

On Wednesday, 13 June 2012 13:06:14 Armin Rigo wrote:
> Hi David,
> 
> On Tue, Jun 12, 2012 at 11:58 AM, David Naylor <naylor.b.david at gmail.com> wrote:
> > [translation:ERROR] .. (pypy.rlib.jit:557)set_param__None_enable_opts
> 
> Nowadays, you cannot use the JIT with the CLI backend.

Translating with -O2 I get:
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 308, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 791, in proceed
[translation:ERROR]     return self._execute(goals, task_skip = self._maybe_skip())
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/tool/taskengine.py", line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 285, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 618, in task_source_cli
[translation:ERROR]     filename = self.gen.generate_source()
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/cli/gencli.py", line 56, in generate_source
[translation:ERROR]     GenOO.generate_source(self)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 68, in generate_source
[translation:ERROR]     self.gen_pendings()
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/genoo.py", line 83, in gen_pendings
[translation:ERROR]     node.render(self.ilasm)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 131, in render
[translation:ERROR]     self.render_normal_block(block)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 239, in render_normal_block
[translation:ERROR]     self._render_op(op)
[translation:ERROR]    File "/tmp/tmp/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/oosupport/function.py", line 369, in _render_op
[translation:ERROR]     assert instr_list is not None, 'Unknown opcode: %s ' % op
[translation:ERROR]  AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120613/c7a321f9/attachment.pgp>

From tobias.oberstein at tavendo.de  Wed Jun 13 16:15:22 2012
From: tobias.oberstein at tavendo.de (Tobias Oberstein)
Date: Wed, 13 Jun 2012 07:15:22 -0700
Subject: [pypy-dev] 300 bucks reward for fixing PyPy/OpenSSL issue
In-Reply-To: <CAGmFidbht=gkYFLfDdwzEkCTfYWyiLRL8YK7vC7VgRj62ryy6Q@mail.gmail.com>
References: <634914A010D0B943A035D226786325D43377FA40B3@EXVMBX020-12.exch020.serverdata.net>
	<CAGmFidbht=gkYFLfDdwzEkCTfYWyiLRL8YK7vC7VgRj62ryy6Q@mail.gmail.com>
Message-ID: <634914A010D0B943A035D226786325D43377FA4199@EXVMBX020-12.exch020.serverdata.net>

>Did you try with a recent nightly build? One created after this change:
>http://mail.python.org/pipermail/pypy-commit/2012-June/063032.html?

Tried, same issue;(



[autobahn at autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$ ~/pypy3/bin/pypy testcase.py ssl pool
adbapi.ConnectionPool created
running SSL on 8090
starting factory
adbapi.ConnectionPool created
connection made
Fatal error: pthread_mutex_unlock(&mutex_gil)
Abort trap: 6

[autobahn at autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$ ~/pypy3/bin/pypy -V
Python 2.7.2 (b5f81e7709a3, Jun 13 2012, 11:19:04)
[PyPy 1.9.1-dev0 with GCC 4.2.1]
[autobahn at autobahnws ~/scm/scratchbox/python/twisted/pypybug1]$

[autobahn at autobahnws ~/scm/pypy]$ hg log | grep -n3 b5f81e7709a3
12-date:        Wed Jun 13 08:09:57 2012 +0200
13-summary:     CPython Issue #13333: The UTF-7 decoder now accepts lone surrogates
14-
15:changeset:   55636:b5f81e7709a3
16-parent:      55631:a53f80a66ef8
17-user:        mattip <matti.picus at gmail.com>
18-date:        Wed Jun 13 08:53:37 2012 +0300


From Ronny.Pfannschmidt at gmx.de  Wed Jun 13 23:29:05 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 13 Jun 2012 23:29:05 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
	<4FD8471E.4040405@gmx.de>
	<CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>
Message-ID: <4FD90621.1050401@gmx.de>

On 06/13/2012 01:20 PM, Armin Rigo wrote:
> Hi Ronny,
>
> On Wed, Jun 13, 2012 at 9:54 AM, Ronny Pfannschmidt
> <Ronny.Pfannschmidt at gmx.de>  wrote:
>> We would appreciate a unified diff or
>> something that can easily be feed to hg import


Sorry, I didn't mean to reject, but to put emphasis on the workflow when 
the patch has a more convient form for our scm its much more easy to 
work with and to attribute it.

I think we should encourage contributors to use tools that help the core 
developers/reviewers whenever possible.

-- Ronny
>
> I don't think I would reject such a patch from these grounds; at least
> it *is* a patch, easily usable with "patch", and not, say, entire
> files that have been modified from some unknown revision, as we got
> too.
>
> I have checked in a slightly edited version of
> http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch (hg
> revision c7dff5469611).  Please tell me if I missed something.
>
>
> A bient?t,
>
> Armin.


From uwe_f_mayer at yahoo.com  Thu Jun 14 01:11:09 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Wed, 13 Jun 2012 16:11:09 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <4FD90621.1050401@gmx.de>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
	<4FD8471E.4040405@gmx.de>
	<CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>
	<4FD90621.1050401@gmx.de>
Message-ID: <1339629069.81110.YahooMailNeo@web162905.mail.bf1.yahoo.com>

> From:?Ronny Pfannschmidt <Ronny.Pfannschmidt at gmx.de>
> I think we should encourage contributors to use tools that help the core?developers/reviewers whenever possible.

Ronny:
I certainly don't mind generating diffs in a format that works for the dev team. It would be nice if that format or desired tools were described somewhere easily found on pypy.org (maybe it's there, but I didn't see it). I can foresee that I may stay involved with pypy, so if I understand correctly, you'd prefer the output of "diff -cru" instead of "diff -cr"?
--Uwe

On 06/13/2012 01:20 PM, Armin Rigo wrote:
> I have checked in a slightly edited version of

> http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch (hg
> revision c7dff5469611).? Please tell me if I missed something.

Armin:
You did not check in the change below (you may have left that one out on purpose, I don't know). If I recall correctly, having write_barrier_failing_case_ptr uninitialized caused an error when compiling with --gc=hybrid --opt=jit. I doubt this is Cygwin specific.
On any case, without the patch below "python translate.py -Ojit" succeeds straight out of the repository under Cygwin with pypy source revision:?1613ac9e0cbd,?tag: tip,?date: Wed Jun 13 16:16:58 2012 +0200. Great!

--Uwe
*** pypy/pypy/rpython/memory/gctransform/framework.py ? Sun Jun ?3 11:01:06 2012
--- ../pypy-work/pypy/pypy/rpython/memory/gctransform/framework.py ? ? ?Tue Jun?
***************
*** 448,453 ****
--- 448,454 ----

? ? ? ? ? self.write_barrier_ptr = None
? ? ? ? ? self.write_barrier_from_array_ptr = None
+ ? ? ? ? self.write_barrier_failing_case_ptr = None
? ? ? ? ? if GCClass.needs_write_barrier:
? ? ? ? ? ? ? self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[s_gc,

From Ronny.Pfannschmidt at gmx.de  Thu Jun 14 01:52:49 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Thu, 14 Jun 2012 01:52:49 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339629069.81110.YahooMailNeo@web162905.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
	<4FD8471E.4040405@gmx.de>
	<CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>
	<4FD90621.1050401@gmx.de>
	<1339629069.81110.YahooMailNeo@web162905.mail.bf1.yahoo.com>
Message-ID: <4FD927D1.5040808@gmx.de>

On 06/14/2012 01:11 AM, Uwe F. Mayer wrote:
>> From: Ronny Pfannschmidt<Ronny.Pfannschmidt at gmx.de>
>> I think we should encourage contributors to use tools that help the core developers/reviewers whenever possible.
>
> Ronny:
> I certainly don't mind generating diffs in a format that works for the dev team. It would be nice if that format or desired tools were described somewhere easily found on pypy.org (maybe it's there, but I didn't see it). I can foresee that I may stay involved with pypy, so if I understand correctly, you'd prefer the output of "diff -cru" instead of "diff -cr"?
> --Uwe
>
basically use mercurial,
i think we need a more detailed howto for setting contributors up for 
sending patches and sending pull requests
(our branching is quite bit more complex
  than the common mercurial project)


> On 06/13/2012 01:20 PM, Armin Rigo wrote:
>> I have checked in a slightly edited version of
>
>> http://www.tux.org/~mayer/cygwin/pypy/pypy-pypy-ac392fb76904.patch (hg
>> revision c7dff5469611).  Please tell me if I missed something.
>
> Armin:
> You did not check in the change below (you may have left that one out on purpose, I don't know). If I recall correctly, having write_barrier_failing_case_ptr uninitialized caused an error when compiling with --gc=hybrid --opt=jit. I doubt this is Cygwin specific.
> On any case, without the patch below "python translate.py -Ojit" succeeds straight out of the repository under Cygwin with pypy source revision: 1613ac9e0cbd, tag: tip, date: Wed Jun 13 16:16:58 2012 +0200. Great!
>
> --Uwe
> *** pypy/pypy/rpython/memory/gctransform/framework.py   Sun Jun  3 11:01:06 2012
> --- ../pypy-work/pypy/pypy/rpython/memory/gctransform/framework.py      Tue Jun
> ***************
> *** 448,453 ****
> --- 448,454 ----
>
>            self.write_barrier_ptr = None
>            self.write_barrier_from_array_ptr = None
> +         self.write_barrier_failing_case_ptr = None
>            if GCClass.needs_write_barrier:
>                self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func,
>                                               [s_gc,


From alexander.belopolsky at gmail.com  Thu Jun 14 02:22:55 2012
From: alexander.belopolsky at gmail.com (Alexander Belopolsky)
Date: Wed, 13 Jun 2012 20:22:55 -0400
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
Message-ID: <CAP7h-xZi0TeHVKbdRtBVhtoF+Bj9hp18eUakwpPq_MfTcHFU5w@mail.gmail.com>

On Mon, Apr 30, 2012 at 7:05 PM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> In particular the work-around for the tm structure to get by without the
> tm_gmtoff and tm_zone fields may be of interest, as it allows compilation of
> the rctime module on systems that don't have it, such as some versions of
> SunOS.

Interesting and timely.  I am about to add limited tm_gmtoff/tm_zone
support to CPython.  See <http://bugs.python.org/issue1667546>.

From uwe_f_mayer at yahoo.com  Thu Jun 14 03:44:33 2012
From: uwe_f_mayer at yahoo.com (Uwe F. Mayer)
Date: Wed, 13 Jun 2012 18:44:33 -0700 (PDT)
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <CAP7h-xZi0TeHVKbdRtBVhtoF+Bj9hp18eUakwpPq_MfTcHFU5w@mail.gmail.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAP7h-xZi0TeHVKbdRtBVhtoF+Bj9hp18eUakwpPq_MfTcHFU5w@mail.gmail.com>
Message-ID: <1339638273.95552.YahooMailNeo@web162902.mail.bf1.yahoo.com>

The workaround I implement is based on full-hour differences between localtime
and gmtime, and uses hardcoded GMT+/-# strings. It obviously will not work correctly
for timezones that have half-hour offsets to GMT, but it seemed to do the trick
for what I needed.


----- Original Message -----
From: Alexander Belopolsky <alexander.belopolsky at gmail.com>
To: Uwe F. Mayer <uwe_f_mayer at yahoo.com>
Cc: "pypy-dev at python.org" <pypy-dev at python.org>
Sent: Wednesday, June 13, 2012 5:22 PM
Subject: Re: [pypy-dev] Patches for Pypy under cygwin

On Mon, Apr 30, 2012 at 7:05 PM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> In particular the work-around for the tm structure to get by without the
> tm_gmtoff and tm_zone fields may be of interest, as it allows compilation of
> the rctime module on systems that don't have it, such as some versions of
> SunOS.

Interesting and timely.? I am about to add limited tm_gmtoff/tm_zone
support to CPython.? See <http://bugs.python.org/issue1667546>.


From csparr at ucsd.edu  Thu Jun 14 23:06:44 2012
From: csparr at ucsd.edu (Cameron Sparr)
Date: Thu, 14 Jun 2012 15:06:44 -0600
Subject: [pypy-dev] Problem linking 64 Bit library when 32 bit one is
 available PyPy 1.9
Message-ID: <4FDA5264.2060400@ucsd.edu>

I am trying to build PyPy 1.9 from source, but getting an error message:

     [platform:Error] /usr/bin/ld: warning: i386 architecture of input 
file `/usr/lib/libcrypto.a(cversion.o)' is incompatible with i386:x86-64 
output

I have this: -L/usr/lib64/ in LDFLAGS and the gcc line is:

     [platform:execute] gcc /tmp/usession-release-1.9-11/platcheck_8.o 
-pthread -L/usr/lib64/ -L/contrib/libffi-3.0.11/lib/ /usr/lib/libssl.a 
/usr/lib/libcrypto.a -lz -lrt -o /tmp/usession-release-1.9-11/platcheck_8

In fact the correct libcrypto.a library would be /usr/lib64/libcrypto.a 
and it is installed on the system. I have seen this IRC log: 
http://tismerysoft.de/pypy/irc-logs/pypy/pypy.2010-10-30.log.html but I 
there is no makefile in /tmp/usession-release-1.9-xx/ and I don't have 
root access so I can't use a chroot environment.

thanks!
Cameron

-- 
Cameron Sparr
SIParCS Intern
NCAR \ CISL \ User Services Section (USS)
phone:  303-497-2443
mobile: 503-327-5964
email:  csparr at ucar.edu


From amauryfa at gmail.com  Thu Jun 14 23:19:35 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 14 Jun 2012 23:19:35 +0200
Subject: [pypy-dev] Problem linking 64 Bit library when 32 bit one is
 available PyPy 1.9
In-Reply-To: <4FDA5264.2060400@ucsd.edu>
References: <4FDA5264.2060400@ucsd.edu>
Message-ID: <CAGmFidbr0jQ7CABbQO6bkxgB84+eJ45aiAKhyJ7N2Y7+kd77Bw@mail.gmail.com>

2012/6/14 Cameron Sparr <csparr at ucsd.edu>

> but I there is no makefile in /tmp/usession-release-1.9-xx/


Actually the Makefile is in the testing_1 subdirectory.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120614/51d08e83/attachment.html>

From set at doretse.com  Thu Jun 14 21:08:04 2012
From: set at doretse.com (Sara Loin)
Date: Thu, 14 Jun 2012 21:08:04 +0200
Subject: [pypy-dev] FWD: 4 Foods Never to Eat
Message-ID: <201206141925.q5EJPxTU017092@smtpweb.clb.oleane.net>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120614/48436b69/attachment.html>

From arigo at tunes.org  Fri Jun 15 07:44:59 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Jun 2012 07:44:59 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <201206131449.11403.naylor.b.david@gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<CAMSv6X3Og_F0f7pBdTA=Uhy07Uxah6pME-H95FGReTmwBz1+8w@mail.gmail.com>
	<201206131449.11403.naylor.b.david@gmail.com>
Message-ID: <CAMSv6X2fmN=001zhUS2dk5+zwdM5aoa57chqm+5kH1AUx0_nrg@mail.gmail.com>

Hi David,

On Wed, Jun 13, 2012 at 2:49 PM, David Naylor <naylor.b.david at gmail.com> wrote:
> [translation:ERROR] ? ? assert instr_list is not None, 'Unknown opcode: %s ' % op
> [translation:ERROR] ?AssertionError: Unknown opcode: v2539 = jit_is_virtual(vref_0)

Try again?

Armin

From arigo at tunes.org  Fri Jun 15 08:14:11 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Jun 2012 08:14:11 +0200
Subject: [pypy-dev] Patches for Pypy under cygwin
In-Reply-To: <1339629069.81110.YahooMailNeo@web162905.mail.bf1.yahoo.com>
References: <1335827155.24082.YahooMailNeo@web162906.mail.bf1.yahoo.com>
	<CAMSv6X3BkQubrrb1w9wyO-=t9_BRg0tGZCOEB=5eH5nQ0zB-9g@mail.gmail.com>
	<1339087466.2170.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidYPmnpUG3ckYRGH+UyuT-0mbtfNU6+WtGDqEM4OYLZ7Qw@mail.gmail.com>
	<1339096884.44059.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<CAMSv6X1a=dBotbZ++a8NqFfKxL0MAAcnf=1Uepc1HAO6ycPx_A@mail.gmail.com>
	<1339284974.43740.YahooMailNeo@web162901.mail.bf1.yahoo.com>
	<CAGmFidaTY-UvhzXyoqCidwP_tyo3WFnAo3q_OaVDpUm-JRsLQg@mail.gmail.com>
	<CAMSv6X1kk9FW_5JYdRROcW_+45bmqbANXb+gfnW6zPBhVo2nRQ@mail.gmail.com>
	<1339435660.42989.YahooMailNeo@web162905.mail.bf1.yahoo.com>
	<1339552211.61761.YahooMailNeo@web162902.mail.bf1.yahoo.com>
	<4FD8471E.4040405@gmx.de>
	<CAMSv6X1jJ9LmY56M5q-X-1PMLLVjougsQw+QCEVaSQSaZ5Pb5g@mail.gmail.com>
	<4FD90621.1050401@gmx.de>
	<1339629069.81110.YahooMailNeo@web162905.mail.bf1.yahoo.com>
Message-ID: <CAMSv6X3MenSMpG1favPkv7srqJmZPkvRsbhdM0O5TJOrZ5=qFw@mail.gmail.com>

Hi Uwe,

On Thu, Jun 14, 2012 at 1:11 AM, Uwe F. Mayer <uwe_f_mayer at yahoo.com> wrote:
> You did not check in the change below (you may have left that one out on purpose, I don't know). If I recall correctly, having write_barrier_failing_case_ptr uninitialized caused an error when compiling with --gc=hybrid --opt=jit. I doubt this is Cygwin specific.

Ah, I see now.  But the error is deeper; your fix merely causes the JIT
to call a NULL function pointer at run-time, as far as I can tell.  We
indeed don't have any test for hybrid+JIT, which means that over time
it just broke.  I "fixed" it by definitely requiring --gc=minimark
with the JIT.


A bient?t,

Armin.

From arigo at tunes.org  Fri Jun 15 08:48:18 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Jun 2012 08:48:18 +0200
Subject: [pypy-dev] Problem linking 64 Bit library when 32 bit one is
 available PyPy 1.9
In-Reply-To: <CAGmFidbr0jQ7CABbQO6bkxgB84+eJ45aiAKhyJ7N2Y7+kd77Bw@mail.gmail.com>
References: <4FDA5264.2060400@ucsd.edu>
	<CAGmFidbr0jQ7CABbQO6bkxgB84+eJ45aiAKhyJ7N2Y7+kd77Bw@mail.gmail.com>
Message-ID: <CAMSv6X0tYDYpRnnAueOsh1fBP8XnBk8UiMOg+TLAyFiEhhD68A@mail.gmail.com>

Hi Cameron, hi Amaury,

On Thu, Jun 14, 2012 at 11:19 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> 2012/6/14 Cameron Sparr <csparr at ucsd.edu>
>> but I there is no makefile in /tmp/usession-release-1.9-xx/
>
> Actually the Makefile is in the testing_1 subdirectory.

Not yet: I would guess that this crash occurs at the start of
translation.  The hard-coded paths /usr/lib/lib{ssl,crypto}.a come
from pypy/rlib/ropenssl.py; see and fix them here.  It's a hack, but a
needed one to avoid the infinite amount of troubles distributing Linux
binaries on systems which can have either openssl0.9.8 or
openssl1.0.0.


A bient?t,

Armin.

From jan at jandecaluwe.com  Fri Jun 15 10:51:59 2012
From: jan at jandecaluwe.com (Jan Decaluwe)
Date: Fri, 15 Jun 2012 10:51:59 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
Message-ID: <4FDAF7AF.1090800@jandecaluwe.com>

For your info.

Since pypy 1.6, the benchmarks for MyHDL have stagnated
or become slightly worse (within 15%).

Still, the results are much better than the average
speedup over the pypy set of benchmarks. I guess it is
to be expected that improvements depend heavily on
a particular application.

http://myhdl.org/doku.php/performance?&#historical_data

Jan Decaluwe

-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     World-class digital design: http://www.easics.com

From grzegorz-rudnicki1 at wp.pl  Fri Jun 15 13:22:31 2012
From: grzegorz-rudnicki1 at wp.pl (Grzegorz Rudnicki)
Date: Fri, 15 Jun 2012 13:22:31 +0200
Subject: [pypy-dev] new benchmark for pypy
Message-ID: <4fdb1af734c414.60345779@wp.pl>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120615/128c6454/attachment.html>

From grzegorz-rudnicki1 at wp.pl  Fri Jun 15 13:30:25 2012
From: grzegorz-rudnicki1 at wp.pl (Grzegorz Rudnicki)
Date: Fri, 15 Jun 2012 13:30:25 +0200
Subject: [pypy-dev] new benchmark for pypy (without html version)
Message-ID: <4fdb1cd18108f7.83094458@wp.pl>

Hi,

I maked new benchmark:

import random,string, time

t=time.time()

def random_string(dl=10):
    d = [random.choice(string.letters) for x in xrange(dl)]
    s = "".join(d)
    return s

basic=[]
words=[]
n=20000   #number of basic words
n2=5000  #number of words in text

#Create list of words variants
for i in range(n):
    basic_word=random_string()
    if basic_word in basic:
        basic_word=random_string()
    basic.append(basic_word)
    tmp=[]
    tmp.append(basic_word)
    for j in range(4):
        tmp.append(basic_word+random_string(3))
    words.append(tmp)

#Create random text
text=[]
for i in range(n2):
    r1=random.randint(0,n-1)
    r2=random.randint(0,3)
    text.append(words[r1][r2])

#Replace non basic word to basic word
for i in range(n2):
    for j in range(n):
        if text[i] in words[j]:
            text[i]=words[j][0]

ok=0
for i in range(n2):
    if text[i] in basic:
        ok+=1

if ok==n2:
    print("OK")
else:
    print("Fail")
print(time.time()-t)

This code tranform non basic word to basic word. For simply I use random strings.

I tested this code on Linux and CPython use 7.8 MB RAM, but Pypy use 14.5 MB RAM.

Grzegorz



From arigo at tunes.org  Fri Jun 15 17:03:53 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 15 Jun 2012 17:03:53 +0200
Subject: [pypy-dev] new benchmark for pypy
In-Reply-To: <4fdb1af734c414.60345779@wp.pl>
References: <4fdb1af734c414.60345779@wp.pl>
Message-ID: <CAMSv6X3Rhy253aaTU592dpWQRsxh1q34CZ2+xhRw6NmVPEh-gg@mail.gmail.com>

Hi Grzegorz,

On Fri, Jun 15, 2012 at 1:22 PM, Grzegorz Rudnicki
<grzegorz-rudnicki1 at wp.pl> wrote:
> I tested this code on Linux and CPython use 7.8 MB RAM, but Pypy use 14.5 MB
> RAM.

That's expected: PyPy uses more memory than CPython in general, with
the exception of programs using a lot of instances --- instances are
more compact in PyPy than in CPython.  Moreover, there is a large
"base size" of some MBs in PyPy.  You need programs consuming a lot
more than 7.8MB to see a possible memory benefit in PyPy, just like
you need programs running for much more than 0.1 seconds to see speed
benefits.


A bient?t,

Armin.

From csparr at ucsd.edu  Fri Jun 15 21:00:44 2012
From: csparr at ucsd.edu (Cameron Sparr)
Date: Fri, 15 Jun 2012 13:00:44 -0600
Subject: [pypy-dev] OSError trying to build 1.9 from source
Message-ID: <4FDB865C.5090302@ucsd.edu>

Hi all,

thanks for all the help so far! Still trying to build PyPy 1.9 on my 
64-bit RHEL system. I've fixed all error messages except for this one 
(translation exits and enters debugger mode), which has me confounded:

     OSError: 
/fs/local/apps/Rational/releases/purify.i386_linux2.7.0.1.0-002/cache/storm0/lib64/libc.so.6_pure_p3_c0_1009241029_2618-194171el5_64: 
undefined symbol: __pure_dl_open

Any ideas? This seems to be related to the problem I was having with the 
pre-built binary (I needed GLIBC_2.7, or libc.so.6).

Thanks,
Cameron

-- 
Cameron Sparr
SIParCS Intern
NCAR \ CISL \ User Services Section (USS)
phone:  303-497-2443
mobile: 503-327-5964
email:  csparr at ucar.edu


From amauryfa at gmail.com  Fri Jun 15 22:50:51 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 15 Jun 2012 22:50:51 +0200
Subject: [pypy-dev] OSError trying to build 1.9 from source
In-Reply-To: <4FDB865C.5090302@ucsd.edu>
References: <4FDB865C.5090302@ucsd.edu>
Message-ID: <CAGmFidap=mn63JBfFmKzik6BvrsaWo0JopfniwJkgx_tLusHhg@mail.gmail.com>

2012/6/15 Cameron Sparr <csparr at ucsd.edu>

> Hi all,
>
> thanks for all the help so far! Still trying to build PyPy 1.9 on my
> 64-bit RHEL system. I've fixed all error messages except for this one
> (translation exits and enters debugger mode), which has me confounded:
>
>    OSError: /fs/local/apps/Rational/**releases/purify.i386_linux2.7.**
> 0.1.0-002/cache/storm0/lib64/**libc.so.6_pure_p3_c0_**1009241029_2618-194171el5_64:
> undefined symbol: __pure_dl_open
>

"Rational", "pure"...
Is there a Purify installation somewhere that interacts incorrectly with
gcc?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120615/708d4d15/attachment.html>

From naylor.b.david at gmail.com  Fri Jun 15 22:55:57 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Fri, 15 Jun 2012 22:55:57 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAMSv6X2fmN=001zhUS2dk5+zwdM5aoa57chqm+5kH1AUx0_nrg@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206131449.11403.naylor.b.david@gmail.com>
	<CAMSv6X2fmN=001zhUS2dk5+zwdM5aoa57chqm+5kH1AUx0_nrg@mail.gmail.com>
Message-ID: <201206152256.00847.naylor.b.david@gmail.com>

On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
> Hi David,
> 
> On Wed, Jun 13, 2012 at 2:49 PM, David Naylor <naylor.b.david at gmail.com> 
wrote:
> > [translation:ERROR]     assert instr_list is not None, 'Unknown opcode:
> > %s ' % op [translation:ERROR]  AssertionError: Unknown opcode: v2539 =
> > jit_is_virtual(vref_0)
> 
> Try again?

Aren't the builds deterministic?  I got a similar result when using -O0 
(although different opcode).  Are you asking me to try the latest for 
mercurial?  I had been using the released sources.  
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120615/82ac8c8f/attachment.pgp>

From fijall at gmail.com  Sat Jun 16 00:34:56 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jun 2012 00:34:56 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <4FDAF7AF.1090800@jandecaluwe.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>
Message-ID: <CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>

On Fri, Jun 15, 2012 at 10:51 AM, Jan Decaluwe <jan at jandecaluwe.com> wrote:

> For your info.
>
> Since pypy 1.6, the benchmarks for MyHDL have stagnated
> or become slightly worse (within 15%).
>
> Still, the results are much better than the average
> speedup over the pypy set of benchmarks. I guess it is
> to be expected that improvements depend heavily on
> a particular application.
>
> http://myhdl.org/doku.php/**performance?&#historical_data<http://myhdl.org/doku.php/performance?&#historical_data>
>
> Jan Decaluwe
>
> --
> Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
>    Python as a HDL: http://www.myhdl.org
>    VHDL development, the modern way: http://www.sigasi.com
>    World-class digital design: http://www.easics.com
> ______________________________**_________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>

I would put the 15% to "noise" honestly, but well. I'm not incredibly
surprised, our optimization are more of the "a bit here a bit there" kind
these days. Maybe the next one :)

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/6b3fba1c/attachment-0001.html>

From fijall at gmail.com  Sat Jun 16 00:35:54 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jun 2012 00:35:54 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>
	<CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>
Message-ID: <CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>

On Sat, Jun 16, 2012 at 12:34 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Fri, Jun 15, 2012 at 10:51 AM, Jan Decaluwe <jan at jandecaluwe.com>wrote:
>
>> For your info.
>>
>> Since pypy 1.6, the benchmarks for MyHDL have stagnated
>> or become slightly worse (within 15%).
>>
>> Still, the results are much better than the average
>> speedup over the pypy set of benchmarks. I guess it is
>> to be expected that improvements depend heavily on
>> a particular application.
>>
>> http://myhdl.org/doku.php/**performance?&#historical_data<http://myhdl.org/doku.php/performance?&#historical_data>
>>
>> Jan Decaluwe
>>
>> --
>> Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
>>    Python as a HDL: http://www.myhdl.org
>>    VHDL development, the modern way: http://www.sigasi.com
>>    World-class digital design: http://www.easics.com
>> ______________________________**_________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>>
>
> I would put the 15% to "noise" honestly, but well. I'm not incredibly
> surprised, our optimization are more of the "a bit here a bit there" kind
> these days. Maybe the next one :)
>
> Cheers,
> fijal
>

Anyway, I'm pleasantly surprised about the high expectations we generate :)
1.6 was about 9 months ago and people already complain there were no
significant speed improvements since then :)))

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/a47b569e/attachment.html>

From fijall at gmail.com  Sat Jun 16 01:29:01 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jun 2012 01:29:01 +0200
Subject: [pypy-dev] OSError trying to build 1.9 from source
In-Reply-To: <CAGmFidap=mn63JBfFmKzik6BvrsaWo0JopfniwJkgx_tLusHhg@mail.gmail.com>
References: <4FDB865C.5090302@ucsd.edu>
	<CAGmFidap=mn63JBfFmKzik6BvrsaWo0JopfniwJkgx_tLusHhg@mail.gmail.com>
Message-ID: <CAK5idxSPKQWtGz14Tz7yS_m7GNjKwZwS8AR=NSrCGCAcQiSprg@mail.gmail.com>

On Fri, Jun 15, 2012 at 10:50 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com>wrote:

>
>
> 2012/6/15 Cameron Sparr <csparr at ucsd.edu>
>
>> Hi all,
>>
>> thanks for all the help so far! Still trying to build PyPy 1.9 on my
>> 64-bit RHEL system. I've fixed all error messages except for this one
>> (translation exits and enters debugger mode), which has me confounded:
>>
>>    OSError: /fs/local/apps/Rational/**releases/purify.i386_linux2.7.**
>> 0.1.0-002/cache/storm0/lib64/**libc.so.6_pure_p3_c0_**1009241029_2618-194171el5_64:
>> undefined symbol: __pure_dl_open
>>
>
> "Rational", "pure"...
> Is there a Purify installation somewhere that interacts incorrectly with
> gcc?
>
> --
> Amaury Forgeot d'Arc
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
Famouse SE linux in RHEL or something?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/ff94bb9f/attachment.html>

From jan at jandecaluwe.com  Sat Jun 16 02:07:56 2012
From: jan at jandecaluwe.com (Jan Decaluwe)
Date: Sat, 16 Jun 2012 02:07:56 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>
	<CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>
	<CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>
Message-ID: <4FDBCE5C.8040500@jandecaluwe.com>

On 06/16/2012 12:35 AM, Maciej Fijalkowski wrote:
> On Sat, Jun 16, 2012 at 12:34 AM, Maciej Fijalkowski
> <fijall at gmail.com <mailto:fijall at gmail.com>> wrote:
>
> On Fri, Jun 15, 2012 at 10:51 AM, Jan Decaluwe <jan at jandecaluwe.com
> <mailto:jan at jandecaluwe.com>> wrote:
>
> For your info.
>
> Since pypy 1.6, the benchmarks for MyHDL have stagnated or become
> slightly worse (within 15%).
>
> Still, the results are much better than the average speedup over the
> pypy set of benchmarks. I guess it is to be expected that
> improvements depend heavily on a particular application.
>
> http://myhdl.org/doku.php/__performance?&#historical_data
> <http://myhdl.org/doku.php/performance?&#historical_data>
>
> Jan Decaluwe
>
> -- Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com Python
> as a HDL: http://www.myhdl.org VHDL development, the modern way:
> http://www.sigasi.com World-class digital design:
> http://www.easics.com
> _________________________________________________ pypy-dev mailing
> list pypy-dev at python.org <mailto:pypy-dev at python.org>
> http://mail.python.org/__mailman/listinfo/pypy-dev
> <http://mail.python.org/mailman/listinfo/pypy-dev>
>
>
> I would put the 15% to "noise" honestly, but well. I'm not incredibly
> surprised, our optimization are more of the "a bit here a bit there"
> kind these days. Maybe the next one :)
>
> Cheers, fijal
>
>
> Anyway, I'm pleasantly surprised about the high expectations we
> generate :) 1.6 was about 9 months ago and people already complain
> there were no significant speed improvements since then :)))

For the record: no complaints at all. I am very grateful
and satisfied with the status. This is a datapoint for those
interested, nothing more and nothing less.

-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     World-class digital design: http://www.easics.com

From fijall at gmail.com  Sat Jun 16 09:15:42 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jun 2012 09:15:42 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <4FDBCE5C.8040500@jandecaluwe.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>
	<CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>
	<CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>
	<4FDBCE5C.8040500@jandecaluwe.com>
Message-ID: <CAK5idxTgHY=Fw_hdx5DqRGknPvHV_aoTuti5ZHa_iBNPKNESow@mail.gmail.com>

On Sat, Jun 16, 2012 at 2:07 AM, Jan Decaluwe <jan at jandecaluwe.com> wrote:

> On 06/16/2012 12:35 AM, Maciej Fijalkowski wrote:
>
>> On Sat, Jun 16, 2012 at 12:34 AM, Maciej Fijalkowski
>> <fijall at gmail.com <mailto:fijall at gmail.com>> wrote:
>>
>> On Fri, Jun 15, 2012 at 10:51 AM, Jan Decaluwe <jan at jandecaluwe.com
>> <mailto:jan at jandecaluwe.com>> wrote:
>>
>> For your info.
>>
>> Since pypy 1.6, the benchmarks for MyHDL have stagnated or become
>> slightly worse (within 15%).
>>
>> Still, the results are much better than the average speedup over the
>> pypy set of benchmarks. I guess it is to be expected that
>> improvements depend heavily on a particular application.
>>
>> http://myhdl.org/doku.php/__**performance?&#historical_data<http://myhdl.org/doku.php/__performance?&#historical_data>
>>
>> <http://myhdl.org/doku.php/**performance?&#historical_data<http://myhdl.org/doku.php/performance?&#historical_data>
>> >
>>
>> Jan Decaluwe
>>
>> -- Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com Python
>> as a HDL: http://www.myhdl.org VHDL development, the modern way:
>> http://www.sigasi.com World-class digital design:
>> http://www.easics.com
>> ______________________________**___________________ pypy-dev mailing
>> list pypy-dev at python.org <mailto:pypy-dev at python.org>
>> http://mail.python.org/__**mailman/listinfo/pypy-dev<http://mail.python.org/__mailman/listinfo/pypy-dev>
>>
>> <http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>> >
>>
>>
>> I would put the 15% to "noise" honestly, but well. I'm not incredibly
>> surprised, our optimization are more of the "a bit here a bit there"
>> kind these days. Maybe the next one :)
>>
>> Cheers, fijal
>>
>>
>> Anyway, I'm pleasantly surprised about the high expectations we
>> generate :) 1.6 was about 9 months ago and people already complain
>> there were no significant speed improvements since then :)))
>>
>
> For the record: no complaints at all. I am very grateful
> and satisfied with the status. This is a datapoint for those
> interested, nothing more and nothing less.
>
>
> --
> Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
>    Python as a HDL: http://www.myhdl.org
>    VHDL development, the modern way: http://www.sigasi.com
>    World-class digital design: http://www.easics.com
>

I did not read it as complaint. Speaking of which - how feasible would be
to add a myhdl benchmark (or a family) to the pypy benchmark suite? It
seems like "a large python application" kind of use case.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/71d16deb/attachment-0001.html>

From fijall at gmail.com  Sat Jun 16 09:18:22 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 16 Jun 2012 09:18:22 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <201206152256.00847.naylor.b.david@gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206131449.11403.naylor.b.david@gmail.com>
	<CAMSv6X2fmN=001zhUS2dk5+zwdM5aoa57chqm+5kH1AUx0_nrg@mail.gmail.com>
	<201206152256.00847.naylor.b.david@gmail.com>
Message-ID: <CAK5idxTU+fCM1HfiGHFo=DoYB8UH8SRk7XxZjrE5B7Hv=cz0Bw@mail.gmail.com>

On Fri, Jun 15, 2012 at 10:55 PM, David Naylor <naylor.b.david at gmail.com>wrote:

> On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
> > Hi David,
> >
> > On Wed, Jun 13, 2012 at 2:49 PM, David Naylor <naylor.b.david at gmail.com>
> wrote:
> > > [translation:ERROR]     assert instr_list is not None, 'Unknown opcode:
> > > %s ' % op [translation:ERROR]  AssertionError: Unknown opcode: v2539 =
> > > jit_is_virtual(vref_0)
> >
> > Try again?
>
> Aren't the builds deterministic?  I got a similar result when using -O0
> (although different opcode).  Are you asking me to try the latest for
> mercurial?  I had been using the released sources.
>
>
That typically means armin has checked something in and you have to update
and try again (I would think).

For completeness sake (not relevant to this mail) - builds are not entirely
deterministic. In edge cases the exact error (or lack thereof) depends on
some dictionary order (the order in which it's seen by the translator),
which in turn might depend on memory locations of objects.

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/ff303467/attachment.html>

From grzegorz-rudnicki1 at wp.pl  Sat Jun 16 11:14:26 2012
From: grzegorz-rudnicki1 at wp.pl (Grzegorz Rudnicki)
Date: Sat, 16 Jun 2012 11:14:26 +0200
Subject: [pypy-dev] Pypy ARM backend and OS
Message-ID: <4fdc4e727b44c1.46525051@wp.pl>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/9ff79aa8/attachment.html>

From davidmenhur at gmail.com  Sat Jun 16 16:03:00 2012
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Sat, 16 Jun 2012 16:03:00 +0200
Subject: [pypy-dev] A proposal about Numpypy missing functions
Message-ID: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>

Hello,

I have seen that there are some missing functions in numpypy that
could be implemented, at least na?vely, very easily. An example of
this is zeros_like, as we have zeros and shape. Another example is the
random module, easy to port using list comprehension and the random
module from the standard library. In any case, based on the benchmarks
I have seen, I think the pypy version would be even faster than the
pure CPython+Numpy.

Would you be interested in a sub-optimal port of those functions? In
my opinion, it is better to have something not great (that could be
improved) that nothing at all. If so, what should I do?


Regards,

David.

From naylor.b.david at gmail.com  Sat Jun 16 16:41:56 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Sat, 16 Jun 2012 16:41:56 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAK5idxTU+fCM1HfiGHFo=DoYB8UH8SRk7XxZjrE5B7Hv=cz0Bw@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206152256.00847.naylor.b.david@gmail.com>
	<CAK5idxTU+fCM1HfiGHFo=DoYB8UH8SRk7XxZjrE5B7Hv=cz0Bw@mail.gmail.com>
Message-ID: <201206161641.59677.naylor.b.david@gmail.com>

On Saturday, 16 June 2012 09:18:22 Maciej Fijalkowski wrote:
> On Fri, Jun 15, 2012 at 10:55 PM, David Naylor 
<naylor.b.david at gmail.com>wrote:
> > On Friday, 15 June 2012 07:44:59 Armin Rigo wrote:
> > > Hi David,
> > > 
> > > On Wed, Jun 13, 2012 at 2:49 PM, David Naylor
> > > <naylor.b.david at gmail.com>
> > 
> > wrote:
> > > > [translation:ERROR]     assert instr_list is not None, 'Unknown
> > > > opcode: %s ' % op [translation:ERROR]  AssertionError: Unknown
> > > > opcode: v2539 = jit_is_virtual(vref_0)
> > > 
> > > Try again?
> > 
> > Aren't the builds deterministic?  I got a similar result when using -O0
> > (although different opcode).  Are you asking me to try the latest for
> > mercurial?  I had been using the released sources.
> 
> That typically means armin has checked something in and you have to update
> and try again (I would think).

Oh, okay.  I don't have access to the latest sources.  I normally work of the 
released sources.  But thanks for looking at this.  

> For completeness sake (not relevant to this mail) - builds are not entirely
> deterministic. In edge cases the exact error (or lack thereof) depends on
> some dictionary order (the order in which it's seen by the translator),
> which in turn might depend on memory locations of objects.

Interesting, thanks.  

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120616/3680666b/attachment.pgp>

From jan at jandecaluwe.com  Sat Jun 16 21:57:24 2012
From: jan at jandecaluwe.com (Jan Decaluwe)
Date: Sat, 16 Jun 2012 21:57:24 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <CAK5idxQG5VdtdakFFAGpGRg=Fer=SXXtwpQ7PgwaV8y-pjPiFA@mail.gmail.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>
	<CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>
	<CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>
	<4FDBCE5C.8040500@jandecaluwe.com>
	<CAK5idxTgHY=Fw_hdx5DqRGknPvHV_aoTuti5ZHa_iBNPKNESow@mail.gmail.com>
	<4FDC7223.5090908@jandecaluwe.com>
	<CAK5idxQG5VdtdakFFAGpGRg=Fer=SXXtwpQ7PgwaV8y-pjPiFA@mail.gmail.com>
Message-ID: <4FDCE524.30507@jandecaluwe.com>

On 06/16/2012 05:20 PM, Maciej Fijalkowski wrote:

> Have a look at our current benchmarks. They're on
> bitbucket.org/pypy/benchmarks <http://bitbucket.org/pypy/benchmarks>.
> Look at the format. Ideally they would execute a few different kinds
> of examples (say 5) each running for about a second?

At only a second for a benchmark, how do you account for
the JIT warming-up phase?

In MyHDL simulations, I have found the JIT warming-up phase
to be quite significant (which is OK, because I get such
good results afterwards).

I run the benchmarks for around 1 minute to remove
the influence of the JIT warming-up phase from the speed-up
factor. Probably you run the benchmarks on a much faster
machine than my i3 laptop, but still.

-- 
Jan Decaluwe - Resources bvba - http://www.jandecaluwe.com
     Python as a HDL: http://www.myhdl.org
     VHDL development, the modern way: http://www.sigasi.com
     World-class digital design: http://www.easics.com


From grzegorz-rudnicki1 at wp.pl  Sun Jun 17 09:04:23 2012
From: grzegorz-rudnicki1 at wp.pl (Grzegorz Rudnicki)
Date: Sun, 17 Jun 2012 09:04:23 +0200
Subject: [pypy-dev] Pypy ARM backend and operating systems
Message-ID: <4fdd81771b31f0.68617908@wp.pl>

Which operating systems Pypy ARM backend will support?

It will support Tizen?

Grzegorz



From arigo at tunes.org  Sun Jun 17 09:32:35 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 17 Jun 2012 09:32:35 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <201206161641.59677.naylor.b.david@gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206152256.00847.naylor.b.david@gmail.com>
	<CAK5idxTU+fCM1HfiGHFo=DoYB8UH8SRk7XxZjrE5B7Hv=cz0Bw@mail.gmail.com>
	<201206161641.59677.naylor.b.david@gmail.com>
Message-ID: <CAMSv6X2fNtPox79nHRK0H3JrU_jt3D4DR-177Hf4-wbdzRspAA@mail.gmail.com>

Hi David,

On Sat, Jun 16, 2012 at 4:41 PM, David Naylor <naylor.b.david at gmail.com> wrote:
> Oh, okay. ?I don't have access to the latest sources. ?I normally work of the
> released sources. ?But thanks for looking at this.

It's going to take us a loooooong time to fix CLI translation if the
process goes: "you find some minor bug in the release, we attempt to
fix it a bit blindly, and then we wait for the next release to know if
it worked and what the next minor bug is" :-)

In case you want to help slightly more actively, you need to get the
sources via Mercurial's command "hg clone
https://bitbucket.org/pypy/pypy" and then keep up-to-date with "hg
pull && hg update".  Or if you don't mind vasting bandwidth you can
download every time the latest source from
https://bitbucket.org/pypy/pypy/get/default.tar.bz2 or
https://bitbucket.org/pypy/pypy/get/default.zip .


A bient?t,

Armin.

From bokr at oz.net  Sun Jun 17 12:02:29 2012
From: bokr at oz.net (Bengt Richter)
Date: Sun, 17 Jun 2012 12:02:29 +0200
Subject: [pypy-dev] Update on MyHDL benchmarks with pypy
In-Reply-To: <4FDCE524.30507@jandecaluwe.com>
References: <4FDAF7AF.1090800@jandecaluwe.com>	<CAK5idxSNhsx3XEuZQ3_-aCZNHUmq5hJNCOvJbOD8bZKDpeFm0g@mail.gmail.com>	<CAK5idxRtccv+veSvaOVTv0NBBfA6O8vsRuGHW6eaFAFUWfhjQQ@mail.gmail.com>	<4FDBCE5C.8040500@jandecaluwe.com>	<CAK5idxTgHY=Fw_hdx5DqRGknPvHV_aoTuti5ZHa_iBNPKNESow@mail.gmail.com>	<4FDC7223.5090908@jandecaluwe.com>	<CAK5idxQG5VdtdakFFAGpGRg=Fer=SXXtwpQ7PgwaV8y-pjPiFA@mail.gmail.com>
	<4FDCE524.30507@jandecaluwe.com>
Message-ID: <4FDDAB35.1000205@oz.net>

On 06/16/2012 09:57 PM Jan Decaluwe wrote:
> On 06/16/2012 05:20 PM, Maciej Fijalkowski wrote:
>
>> Have a look at our current benchmarks. They're on
>> bitbucket.org/pypy/benchmarks<http://bitbucket.org/pypy/benchmarks>.
>> Look at the format. Ideally they would execute a few different kinds
>> of examples (say 5) each running for about a second?
>
> At only a second for a benchmark, how do you account for
> the JIT warming-up phase?
>
> In MyHDL simulations, I have found the JIT warming-up phase
> to be quite significant (which is OK, because I get such
> good results afterwards).
>
> I run the benchmarks for around 1 minute to remove
> the influence of the JIT warming-up phase from the speed-up
> factor. Probably you run the benchmarks on a much faster
> machine than my i3 laptop, but still.
>
Re faster and slower machines, I wonder if it wouldn't be interesting
to have an option to report results in gigabogoinst instead of,
or in addition to, seconds, (by multiplying seconds by bogogips (== bogomips/1000 ;-)), e.g.,

 >>> bogogips=(lambda b=[float(line.split(':')[1].strip()) for line in open('/proc/cpuinfo') if 
line.startswith('bogomips')]:sum(b)/len(b))()/1000.0
 >>> bogogips
3.99242
would be the factor on my laptop, averaging the bogomips of all (two) CPUs.

If the jit processing had optional callback hooks returning ('<hook id>', time.time()) that
a detailed benchmarker could use to see what activities were taking what time, that would be cool too.
Maybe you already have something like that?

Regards,
Bengt Richter


From timonator at perpetuum-immobile.de  Sun Jun 17 20:46:30 2012
From: timonator at perpetuum-immobile.de (Timo Paulssen)
Date: Sun, 17 Jun 2012 20:46:30 +0200
Subject: [pypy-dev] A proposal about Numpypy missing functions
In-Reply-To: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>
References: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>
Message-ID: <4FDE2606.7080905@perpetuum-immobile.de>

On 16.06.2012 16:03, Da?id wrote:
>  Another example is the random module, easy to port using list
>  comprehension and the random module from the standard library. In
>  any case, based on the benchmarks I have seen, I think the pypy
>  version would be even faster than the pure CPython+Numpy.
>
>  Would you be interested in a sub-optimal port of those functions? In
>  my opinion, it is better to have something not great (that could be
>  improved) that nothing at all. If so, what should I do?
>

Hello Da?id,

I remember having written a random module for numpy based on stdlibs random 
module, but somehow it didn't make it into numpypy. Does anyone know where it went?

Writing that is a bit problematic, though, unless you know how to make all those 
distributions work based on a stream of random bits. I didn't at that time.

>
>  Regards,
>
>  David.

Regards,
   - Timo

From fijall at gmail.com  Mon Jun 18 10:56:14 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 18 Jun 2012 10:56:14 +0200
Subject: [pypy-dev] Pypy ARM backend and operating systems
In-Reply-To: <4fdd81771b31f0.68617908@wp.pl>
References: <4fdd81771b31f0.68617908@wp.pl>
Message-ID: <CAK5idxSsMGrbV-kDPrHL7kSiZ3rsY91fSouzznwi_BBiJqP0EA@mail.gmail.com>

On Sun, Jun 17, 2012 at 9:04 AM, Grzegorz Rudnicki <grzegorz-rudnicki1 at wp.pl
> wrote:

> Which operating systems Pypy ARM backend will support?
>
> It will support Tizen?
>
> Grzegorz
>

Hi

I think for now there is linux support, but nothing stops you from
submitting patches (it should not be too hard)

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/0943ba2f/attachment.html>

From fijall at gmail.com  Mon Jun 18 11:01:36 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 18 Jun 2012 11:01:36 +0200
Subject: [pypy-dev] A proposal about Numpypy missing functions
In-Reply-To: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>
References: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>
Message-ID: <CAK5idxRFOHQHaMCcLuT7Pw4Bca2C_GEs-NpkqYhyO_v_YBUKeA@mail.gmail.com>

On Sat, Jun 16, 2012 at 4:03 PM, Da?id <davidmenhur at gmail.com> wrote:

> Hello,
>
> I have seen that there are some missing functions in numpypy that
> could be implemented, at least na?vely, very easily. An example of
> this is zeros_like, as we have zeros and shape. Another example is the
> random module, easy to port using list comprehension and the random
> module from the standard library. In any case, based on the benchmarks
> I have seen, I think the pypy version would be even faster than the
> pure CPython+Numpy.
>
> Would you be interested in a sub-optimal port of those functions? In
> my opinion, it is better to have something not great (that could be
> improved) that nothing at all. If so, what should I do?
>

The only requirement is that they come with tests (including weird corner
cases)

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/7ecf8606/attachment.html>

From robert.zaremba at zoho.com  Mon Jun 18 13:02:23 2012
From: robert.zaremba at zoho.com (Robert Zaremba)
Date: Mon, 18 Jun 2012 13:02:23 +0200
Subject: [pypy-dev] HTMLParser  compatibility with cPython 2.7.3
Message-ID: <2403998.ioWsBH0rbP@robert-zaremba-bc>

Hi, I would like to import changes from:
The problem is that HTMLParser from 2.7.2 is not lenient and likes to throw 
exceptions, when html document is not well formed: 
http://bugs.python.org/issue13987

This often involves exception from BeautifoulSoup, which gains great speed up 
when using from pypy + HTMLParser from stdlib:
    "RuntimeWarning: Python's built-in HTMLParser cannot parse the given 
document. This is not a bug in Beautiful Soup. The best solution is to install 
an external parser (lxml or html5lib), and use Beautiful Soup with that 
parser. See http://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-
a-parser for help."

However lxml is not compatibile with PyPy, and html5lib is slow.

Can I port the HTMLParser.py from python 2.7.3 to PyPy?

-- 
  Robert Zaremba


From amauryfa at gmail.com  Mon Jun 18 14:17:21 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 18 Jun 2012 14:17:21 +0200
Subject: [pypy-dev] HTMLParser compatibility with cPython 2.7.3
In-Reply-To: <2403998.ioWsBH0rbP@robert-zaremba-bc>
References: <2403998.ioWsBH0rbP@robert-zaremba-bc>
Message-ID: <CAGmFidatGXUCGspM-ZX-rawPVT2Hg7YTnyKc9LHQKhiWm2yeuw@mail.gmail.com>

2012/6/18 Robert Zaremba <robert.zaremba at zoho.com>

> Hi, I would like to import changes from:
> The problem is that HTMLParser from 2.7.2 is not lenient and likes to throw
> exceptions, when html document is not well formed:
> http://bugs.python.org/issue13987
>
> This often involves exception from BeautifoulSoup, which gains great speed
> up
> when using from pypy + HTMLParser from stdlib:
>    "RuntimeWarning: Python's built-in HTMLParser cannot parse the given
> document. This is not a bug in Beautiful Soup. The best solution is to
> install
> an external parser (lxml or html5lib), and use Beautiful Soup with that
> parser. See
> http://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-
> a-parser for help."
>
> However lxml is not compatibile with PyPy, and html5lib is slow.
>
> Can I port the HTMLParser.py from python 2.7.3 to PyPy?
>

In general, no, unless you port the all the rest to 2.7.3 as well.
There is already work in progress for this, in the stdlib-2.7.3 branch.

It's almost finished (and definitely worth a try),
there are some nightly builds there (only 32bit Linux for the moment):
http://buildbot.pypy.org/nightly/stdlib-2.7.3/

Still missing are the implementation of randomized hashes (not enabled by
default anyway)
and a couple of obscure bugs in the import system, probably implementation
details.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/e2fd18ab/attachment.html>

From santagada at gmail.com  Mon Jun 18 15:09:37 2012
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 18 Jun 2012 10:09:37 -0300
Subject: [pypy-dev] new benchmark for pypy (without html version)
In-Reply-To: <4fdb1cd18108f7.83094458@wp.pl>
References: <4fdb1cd18108f7.83094458@wp.pl>
Message-ID: <CA+HFc84qXJZhUmzHfWzib67oULb8OYdwLnnEpKVpQK6YqRzDXw@mail.gmail.com>

I know this sounds stupid but isn't using random and generating the
benchmark data completely against the idea of a benchmark? This
doesn't look like it takes the same time to run at each invocation so
how do you compare different runs?

On Fri, Jun 15, 2012 at 8:30 AM, Grzegorz Rudnicki
<grzegorz-rudnicki1 at wp.pl> wrote:
> Hi,
>
> I maked new benchmark:
>
> import random,string, time
>
> t=time.time()
>
> def random_string(dl=10):
> ? ?d = [random.choice(string.letters) for x in xrange(dl)]
> ? ?s = "".join(d)
> ? ?return s
>
> basic=[]
> words=[]
> n=20000 ? #number of basic words
> n2=5000 ?#number of words in text
>
> #Create list of words variants
> for i in range(n):
> ? ?basic_word=random_string()
> ? ?if basic_word in basic:
> ? ? ? ?basic_word=random_string()
> ? ?basic.append(basic_word)
> ? ?tmp=[]
> ? ?tmp.append(basic_word)
> ? ?for j in range(4):
> ? ? ? ?tmp.append(basic_word+random_string(3))
> ? ?words.append(tmp)
>
> #Create random text
> text=[]
> for i in range(n2):
> ? ?r1=random.randint(0,n-1)
> ? ?r2=random.randint(0,3)
> ? ?text.append(words[r1][r2])
>
> #Replace non basic word to basic word
> for i in range(n2):
> ? ?for j in range(n):
> ? ? ? ?if text[i] in words[j]:
> ? ? ? ? ? ?text[i]=words[j][0]
>
> ok=0
> for i in range(n2):
> ? ?if text[i] in basic:
> ? ? ? ?ok+=1
>
> if ok==n2:
> ? ?print("OK")
> else:
> ? ?print("Fail")
> print(time.time()-t)
>
> This code tranform non basic word to basic word. For simply I use random strings.
>
> I tested this code on Linux and CPython use 7.8 MB RAM, but Pypy use 14.5 MB RAM.
>
> Grzegorz
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev



-- 

Leonardo Santagada

From fijall at gmail.com  Mon Jun 18 15:12:13 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 18 Jun 2012 15:12:13 +0200
Subject: [pypy-dev] A very easy bug for newcomers
Message-ID: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>

Hi

This is a good start for someone wanting to explore contributing to pypy -
https://bugs.pypy.org/issue1180

I'll fix it if noone steps up :)

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/4b856440/attachment.html>

From kostia.lopuhin at gmail.com  Mon Jun 18 15:19:05 2012
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Mon, 18 Jun 2012 17:19:05 +0400
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
Message-ID: <CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>

Hello!
May I try? :)

So I should:
* reproduce the bug
* make a branch of default PyPy
* commit the test
* commit the fix (hopefully)
* send pull request



2012/6/18 Maciej Fijalkowski <fijall at gmail.com>:
> Hi
>
> This is a good start for someone wanting to explore contributing to pypy
> -?https://bugs.pypy.org/issue1180
>
> I'll fix it if noone steps up :)
>
> Cheers,
> fijal
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

From amauryfa at gmail.com  Mon Jun 18 15:36:51 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 18 Jun 2012 15:36:51 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
Message-ID: <CAGmFidYCwLb2WKzLVKQGgQnVUD4dmNjj220TvXUMq9azPMXhxQ@mail.gmail.com>

2012/6/18 ????? ??????? <kostia.lopuhin at gmail.com>

> Hello!
> May I try? :)
>

Welcome!

So I should:
> * reproduce the bug
> * make a branch of default PyPy
> * commit the test
> * commit the fix (hopefully)
> * send pull request
>

Looks good. Feel free to hang on IRC (#pypy in irc.freenode.net)
for fast questions/answers...

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/768e6f07/attachment-0001.html>

From stefan_ml at behnel.de  Mon Jun 18 15:53:47 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 18 Jun 2012 15:53:47 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
Message-ID: <jrnbtb$h5o$1@dough.gmane.org>

Hi,

I'm getting reproducible segfaults in the PyThreadState_GetDict() function
with lxml. For that, it's enough to build the latest lxml github master
version in PyPy and start the test runner. You'll also need the github
master version of Cython for that (with contains the PyPy port).

I use these commands for building lxml under Linux:

$ CFLAGS=-ggdb make PYTHON=pypy inplace
$ pypy test.py -vv -p parser html    # no specific test, just not all

The stack trace I get starts like this:

"""
#0  0x0000000000b4fac5 in ?? ()
#1  0x000000000103a813 in PyThreadState_GetDict ()
#2  0x00007ffff51b15b0 in __pyx_f_4lxml_5etree__getGlobalErrorLog () at
src/lxml/lxml.etree.c:32906
#3  0x00007ffff51a8922 in __pyx_f_4lxml_5etree_13_BaseErrorLog__receive
(__pyx_v_self=0x7ffff7f99378, __pyx_v_error=0x3205ed8) at
src/lxml/lxml.etree.c:28299
#4  0x00007ffff51fad46 in __pyx_f_4lxml_5etree__forwardParserError
(__pyx_v__parser_context=0x3205c80, __pyx_v_error=0x3205ed8) at
src/lxml/lxml.etree.c:82070
#5  0x00007ffff51fadc7 in __pyx_f_4lxml_5etree__receiveParserError
(__pyx_v_c_context=0x3205c80, __pyx_v_error=0x3205ed8) at
src/lxml/lxml.etree.c:82126
"""

Background: lxml frees the GIL when it starts parsing XML, then reacquires
it to report any errors that occur and, while holding it, looks up the
target error log object from thread global storage to find out who wants to
know about the error. That's where it crashes above, right when it tries to
get at the TLS dict.

Any help is appreciated.

Stefan


From amauryfa at gmail.com  Mon Jun 18 16:02:30 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 18 Jun 2012 16:02:30 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
In-Reply-To: <jrnbtb$h5o$1@dough.gmane.org>
References: <jrnbtb$h5o$1@dough.gmane.org>
Message-ID: <CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>

2012/6/18 Stefan Behnel <stefan_ml at behnel.de>

> Hi,
>
> I'm getting reproducible segfaults in the PyThreadState_GetDict() function
> with lxml.


Is it similar to https://bugs.pypy.org/issue1175 ?
"PyThread_{get, set, delete}_key_value should work without the GIL held"


> For that, it's enough to build the latest lxml github master
> version in PyPy and start the test runner. You'll also need the github
> master version of Cython for that (with contains the PyPy port).
>
> I use these commands for building lxml under Linux:
>
> $ CFLAGS=-ggdb make PYTHON=pypy inplace
> $ pypy test.py -vv -p parser html    # no specific test, just not all
>
> The stack trace I get starts like this:
>
> """
> #0  0x0000000000b4fac5 in ?? ()
> #1  0x000000000103a813 in PyThreadState_GetDict ()
> #2  0x00007ffff51b15b0 in __pyx_f_4lxml_5etree__getGlobalErrorLog () at
> src/lxml/lxml.etree.c:32906
> #3  0x00007ffff51a8922 in __pyx_f_4lxml_5etree_13_BaseErrorLog__receive
> (__pyx_v_self=0x7ffff7f99378, __pyx_v_error=0x3205ed8) at
> src/lxml/lxml.etree.c:28299
> #4  0x00007ffff51fad46 in __pyx_f_4lxml_5etree__forwardParserError
> (__pyx_v__parser_context=0x3205c80, __pyx_v_error=0x3205ed8) at
> src/lxml/lxml.etree.c:82070
> #5  0x00007ffff51fadc7 in __pyx_f_4lxml_5etree__receiveParserError
> (__pyx_v_c_context=0x3205c80, __pyx_v_error=0x3205ed8) at
> src/lxml/lxml.etree.c:82126
> """
>
> Background: lxml frees the GIL when it starts parsing XML, then reacquires
> it to report any errors that occur and, while holding it, looks up the
> target error log object from thread global storage to find out who wants to
> know about the error. That's where it crashes above, right when it tries to
> get at the TLS dict.
>
> Any help is appreciated.
>
> Stefan
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>



-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/444aea0d/attachment.html>

From stefan_ml at behnel.de  Mon Jun 18 16:14:42 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 18 Jun 2012 16:14:42 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
In-Reply-To: <CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>
References: <jrnbtb$h5o$1@dough.gmane.org>
	<CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>
Message-ID: <jrnd4j$s5g$1@dough.gmane.org>

Amaury Forgeot d'Arc, 18.06.2012 16:02:
> 2012/6/18 Stefan Behnel
>> I'm getting reproducible segfaults in the PyThreadState_GetDict() function
>> with lxml.
> 
> Is it similar to https://bugs.pypy.org/issue1175 ?
> "PyThread_{get, set, delete}_key_value should work without the GIL held"

No. As I said, it has already acquired the GIL when it calls that function.

Stefan


From arigo at tunes.org  Mon Jun 18 16:20:30 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 18 Jun 2012 16:20:30 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
Message-ID: <CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>

Hi Konstantin,

On Mon, Jun 18, 2012 at 3:19 PM, ????? ??????? <kostia.lopuhin at gmail.com> wrote:
> May I try? :)

Sure :-)  A first quick remark: you should add a test in pypy's own
tests, in pypy/objspace/std/test/test_newformat.py.  Then run it with
"python pytest.py pypy/obj.../test_newformat.py".  In this way, you
see the failing test and your fixes immediately, without needing to
wait for a complete translation.


A bient?t,

Armin.

From kostia.lopuhin at gmail.com  Mon Jun 18 16:27:01 2012
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Mon, 18 Jun 2012 18:27:01 +0400
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
Message-ID: <CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>

Ah, I see, thank you! I tried added a test to
lib-python/2.7/test/test_bool.py, but could not run it using
untranslated pypy

2012/6/18 Armin Rigo <arigo at tunes.org>:
> Hi Konstantin,
>
> On Mon, Jun 18, 2012 at 3:19 PM, ????? ??????? <kostia.lopuhin at gmail.com> wrote:
>> May I try? :)
>
> Sure :-) ?A first quick remark: you should add a test in pypy's own
> tests, in pypy/objspace/std/test/test_newformat.py. ?Then run it with
> "python pytest.py pypy/obj.../test_newformat.py". ?In this way, you
> see the failing test and your fixes immediately, without needing to
> wait for a complete translation.
>
>
> A bient?t,
>
> Armin.

From amauryfa at gmail.com  Mon Jun 18 16:51:49 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 18 Jun 2012 16:51:49 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
In-Reply-To: <jrnd4j$s5g$1@dough.gmane.org>
References: <jrnbtb$h5o$1@dough.gmane.org>
	<CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>
	<jrnd4j$s5g$1@dough.gmane.org>
Message-ID: <CAGmFidZgxggcefdCW+g4SR+Uh0+ogO9H9qm5B4-W3S0rwc-9kw@mail.gmail.com>

2012/6/18 Stefan Behnel <stefan_ml at behnel.de>

> Amaury Forgeot d'Arc, 18.06.2012 16:02:
> > 2012/6/18 Stefan Behnel
> >> I'm getting reproducible segfaults in the PyThreadState_GetDict()
> function
> >> with lxml.
> >
> > Is it similar to https://bugs.pypy.org/issue1175 ?
> > "PyThread_{get, set, delete}_key_value should work without the GIL held"
>
> No. As I said, it has already acquired the GIL when it calls that function.


OK, the answer is quite simple:
PyGILState_Ensure() is not really implemented and just returns zero.
:-(

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/2ceebc24/attachment-0001.html>

From stefan_ml at behnel.de  Mon Jun 18 19:04:48 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 18 Jun 2012 19:04:48 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
In-Reply-To: <CAGmFidZgxggcefdCW+g4SR+Uh0+ogO9H9qm5B4-W3S0rwc-9kw@mail.gmail.com>
References: <jrnbtb$h5o$1@dough.gmane.org>
	<CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>
	<jrnd4j$s5g$1@dough.gmane.org>
	<CAGmFidZgxggcefdCW+g4SR+Uh0+ogO9H9qm5B4-W3S0rwc-9kw@mail.gmail.com>
Message-ID: <jrnn3g$jj8$1@dough.gmane.org>

Amaury Forgeot d'Arc, 18.06.2012 16:51:
> 2012/6/18 Stefan Behnel
>> Amaury Forgeot d'Arc, 18.06.2012 16:02:
>>> 2012/6/18 Stefan Behnel
>>>> I'm getting reproducible segfaults in the PyThreadState_GetDict()
>>>> function with lxml.
>>>
>>> Is it similar to https://bugs.pypy.org/issue1175 ?
>>> "PyThread_{get, set, delete}_key_value should work without the GIL held"
>>
>> No. As I said, it has already acquired the GIL when it calls that function.
> 
> OK, the answer is quite simple:
> PyGILState_Ensure() is not really implemented and just returns zero.

Ah, hmmm. That's annoying. Threading is quite crucial for many use cases of
lxml. Is there a chance that this will become available soonish, or should
I just compile out threading support when building in PyPy? The code for
that is there anyway (although I haven't tried it in years...).

Maybe PyPy should undefine the WITH_THREAD macro if it doesn't support
threading in C extensions anyway?

Stefan


From davidmenhur at gmail.com  Mon Jun 18 19:37:55 2012
From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=)
Date: Mon, 18 Jun 2012 19:37:55 +0200
Subject: [pypy-dev] A proposal about Numpypy missing functions
In-Reply-To: <CAK5idxRFOHQHaMCcLuT7Pw4Bca2C_GEs-NpkqYhyO_v_YBUKeA@mail.gmail.com>
References: <CAJhcF=3gbT-yRCW1s7PHv3W+v1ghVDZGV=Uf42PBE-wJ2y-6Fg@mail.gmail.com>
	<CAK5idxRFOHQHaMCcLuT7Pw4Bca2C_GEs-NpkqYhyO_v_YBUKeA@mail.gmail.com>
Message-ID: <CAJhcF=37uAqBA2_v079_L2BcKXe-Fzw=ryyunKOSu4DXTo87HA@mail.gmail.com>

Ok, I will start working on that. Hopefully, you will hear of me again.

On Mon, Jun 18, 2012 at 11:01 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Sat, Jun 16, 2012 at 4:03 PM, Da?id <davidmenhur at gmail.com> wrote:
>>
>> Hello,
>>
>> I have seen that there are some missing functions in numpypy that
>> could be implemented, at least na?vely, very easily. An example of
>> this is zeros_like, as we have zeros and shape. Another example is the
>> random module, easy to port using list comprehension and the random
>> module from the standard library. In any case, based on the benchmarks
>> I have seen, I think the pypy version would be even faster than the
>> pure CPython+Numpy.
>>
>> Would you be interested in a sub-optimal port of those functions? In
>> my opinion, it is better to have something not great (that could be
>> improved) that nothing at all. If so, what should I do?
>
>
> The only requirement is that they come with tests (including weird corner
> cases)
>
> Cheers,
> fijal
>

From allandouglas at gmail.com  Mon Jun 18 19:57:16 2012
From: allandouglas at gmail.com (Allan Douglas R. de Oliveira)
Date: Mon, 18 Jun 2012 14:57:16 -0300
Subject: [pypy-dev] new benchmark for pypy (without html version)
In-Reply-To: <CA+HFc84qXJZhUmzHfWzib67oULb8OYdwLnnEpKVpQK6YqRzDXw@mail.gmail.com>
References: <4fdb1cd18108f7.83094458@wp.pl>
	<CA+HFc84qXJZhUmzHfWzib67oULb8OYdwLnnEpKVpQK6YqRzDXw@mail.gmail.com>
Message-ID: <CAGBFyfOs2qte7k2mRCs9x5Pi5DFizNzgdsHKchi8MtYT2ttQtQ@mail.gmail.com>

I think it's just a matter of setting a fixed seed for the test.

On Mon, Jun 18, 2012 at 10:09 AM, Leonardo Santagada
<santagada at gmail.com> wrote:
> I know this sounds stupid but isn't using random and generating the
> benchmark data completely against the idea of a benchmark? This
> doesn't look like it takes the same time to run at each invocation so
> how do you compare different runs?

-- 
Allan Douglas R. de Oliveira

From stefan_ml at behnel.de  Mon Jun 18 20:25:30 2012
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 18 Jun 2012 20:25:30 +0200
Subject: [pypy-dev] [cpyext] crash in PyThreadState_GetDict()
In-Reply-To: <jrnn3g$jj8$1@dough.gmane.org>
References: <jrnbtb$h5o$1@dough.gmane.org>
	<CAGmFidbV+4DwYg-aK_8VHeL7__eUZiikAg6dTEEMT0VA3J38LA@mail.gmail.com>
	<jrnd4j$s5g$1@dough.gmane.org>
	<CAGmFidZgxggcefdCW+g4SR+Uh0+ogO9H9qm5B4-W3S0rwc-9kw@mail.gmail.com>
	<jrnn3g$jj8$1@dough.gmane.org>
Message-ID: <jrnrqq$qgd$1@dough.gmane.org>

Stefan Behnel, 18.06.2012 19:04:
> Amaury Forgeot d'Arc, 18.06.2012 16:51:
>> 2012/6/18 Stefan Behnel
>>> Amaury Forgeot d'Arc, 18.06.2012 16:02:
>>>> 2012/6/18 Stefan Behnel
>>>>> I'm getting reproducible segfaults in the PyThreadState_GetDict()
>>>>> function with lxml.
>>>>
>>>> Is it similar to https://bugs.pypy.org/issue1175 ?
>>>> "PyThread_{get, set, delete}_key_value should work without the GIL held"
>>>
>>> No. As I said, it has already acquired the GIL when it calls that function.
>>
>> OK, the answer is quite simple:
>> PyGILState_Ensure() is not really implemented and just returns zero.
> 
> Ah, hmmm. That's annoying. Threading is quite crucial for many use cases of
> lxml. Is there a chance that this will become available soonish, or should
> I just compile out threading support when building in PyPy? The code for
> that is there anyway (although I haven't tried it in years...).

I disabled threading and it fixes a lot of crashes. Thanks for the hint.


> Maybe PyPy should undefine the WITH_THREAD macro if it doesn't support
> threading in C extensions anyway?

... or rather, behave as if WITH_THREAD was undefined and make the GIL
handling macros dummies.

Stefan


From arigo at tunes.org  Mon Jun 18 22:40:34 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 18 Jun 2012 22:40:34 +0200
Subject: [pypy-dev] CFFI release 0.1
Message-ID: <CAMSv6X0=UguNKtZbvyEmcUbetdYq9s3h4y=jwsi=E4znCXRH-Q@mail.gmail.com>

Hi all,

Announcing the release 0.1 of CFFI, the new Python interface to call C
libraries done (mostly) by Maciek and myself:

http://morepypy.blogspot.ch/2012/06/release-01-of-cffi.html


A bient?t,

Armin.

From kostia.lopuhin at gmail.com  Mon Jun 18 22:41:49 2012
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Tue, 19 Jun 2012 00:41:49 +0400
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
Message-ID: <CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>

Please see pull request - not sure if I should have added test in
lib-python, because it tests the same thing, and is harder to run?
https://bitbucket.org/pypy/pypy/pull-request/69/fix-issue-1180-implement-bool-formatting

2012/6/18 ????? ??????? <kostia.lopuhin at gmail.com>:
> Ah, I see, thank you! I tried added a test to
> lib-python/2.7/test/test_bool.py, but could not run it using
> untranslated pypy
>
> 2012/6/18 Armin Rigo <arigo at tunes.org>:
>> Hi Konstantin,
>>
>> On Mon, Jun 18, 2012 at 3:19 PM, ????? ??????? <kostia.lopuhin at gmail.com> wrote:
>>> May I try? :)
>>
>> Sure :-) ?A first quick remark: you should add a test in pypy's own
>> tests, in pypy/objspace/std/test/test_newformat.py. ?Then run it with
>> "python pytest.py pypy/obj.../test_newformat.py". ?In this way, you
>> see the failing test and your fixes immediately, without needing to
>> wait for a complete translation.
>>
>>
>> A bient?t,
>>
>> Armin.

From fijall at gmail.com  Mon Jun 18 22:54:10 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 18 Jun 2012 22:54:10 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
Message-ID: <CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>

On Mon, Jun 18, 2012 at 10:41 PM, ????? ??????? <kostia.lopuhin at gmail.com>wrote:

> Please see pull request - not sure if I should have added test in
> lib-python, because it tests the same thing, and is harder to run?
>
> https://bitbucket.org/pypy/pypy/pull-request/69/fix-issue-1180-implement-bool-formatting
>
> 2012/6/18 ????? ??????? <kostia.lopuhin at gmail.com>:
> > Ah, I see, thank you! I tried added a test to
> > lib-python/2.7/test/test_bool.py, but could not run it using
> > untranslated pypy
> >
> > 2012/6/18 Armin Rigo <arigo at tunes.org>:
> >> Hi Konstantin,
> >>
> >> On Mon, Jun 18, 2012 at 3:19 PM, ????? ??????? <
> kostia.lopuhin at gmail.com> wrote:
> >>> May I try? :)
> >>
> >> Sure :-)  A first quick remark: you should add a test in pypy's own
> >> tests, in pypy/objspace/std/test/test_newformat.py.  Then run it with
> >> "python pytest.py pypy/obj.../test_newformat.py".  In this way, you
> >> see the failing test and your fixes immediately, without needing to
> >> wait for a complete translation.
> >>
> >>
> >> A bient?t,
> >>
> >> Armin.
>

Hi Kostia

You should not add tests to lib-python, unless you're going to submit
patches to cpython anyway. It seems that you should make simpler tests (as
in there is no need for self.s and self.b I think, why not use str and bool
builtins?)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120618/5c8d2693/attachment-0001.html>

From arigo at tunes.org  Mon Jun 18 23:00:08 2012
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 18 Jun 2012 23:00:08 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
Message-ID: <CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>

Hi,

On Mon, Jun 18, 2012 at 10:54 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> It seems that you should make simpler tests (as
> in there is no need for self.s and self.b I think, why not use str and bool
> builtins?)

Ah, it came from the other tests of test_newformat.py; but these use
"self.s" instead of "str or unicode".  "self.b()" is superflous, as
you can use "bool()".


A bient?t,

Armin.

From leonard.de.haro at ens.fr  Tue Jun 19 01:41:24 2012
From: leonard.de.haro at ens.fr (Leonard de HARO)
Date: Tue, 19 Jun 2012 01:41:24 +0200
Subject: [pypy-dev] Internship and bug report
Message-ID: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>

Hello,

I'm a french undergraduated CS. student currently doing an internship in
Santigo's University under the supervision of Eric Tanter. I'm working on
the meta JIT compiler provided in the pypy project, trying to document
it's performance on different kind of interpreters. I started two weeks
ago.

I have a problem with the translation toolchain. I've been writing an
interpreter for a small language inspired by Shriram Krishnamurthi's F1WAE
from his book "Programming Languages: Application and Interpretation".
My problem is the following: I have an interpreter that seems to work
perfecty (at least on my examples) when I use pypy interpreters, but
whenever I translate it, execution fails. The fact is: I have identified
the source of the problem but I don't understand it.
If you look at the parser file, on line 184, there is a "print"
instruction. With this instruction on comment, the translated version of
RPinterpret executed on test10run10 gets a Stack Overflow. Use a
translated version of the same file (RPinterpret.py) with the print
instruction in parser.py, you get the normal behavior (which you have
anyway using "pypy RPinterpret.py test10runs10").

Looks like a bug, but since I'm a beginer (both on this work and on Python
in general), maybe I have missed something.

I have the same problem with the supposely-JITing interpreter, but it's
worse since the translations takes 20min against 1min, so changing and
experiencing on the JITing file seems much harder.

Anyway, thanks for your help.
I didn't know whether I was supposed to make a bug report or just contact
the mailing list, but since I'll be working on this until september, might
as well get in touch with you right now.

Leonard de HARO

PS: As I said, I just learned Python, so maybe my files aren't verry
pretty, I apologize for that and I'll welcome any comment on my work you
could make. Thanks again.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: parser.py
Type: text/x-python-script
Size: 10679 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/5559ad5c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: treeClass.py
Type: text/x-python-script
Size: 1827 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/5559ad5c/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RPinterpret.py
Type: text/x-python-script
Size: 2871 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/5559ad5c/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test10runs10
Type: application/octet-stream
Size: 194 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/5559ad5c/attachment.obj>

From kostia.lopuhin at gmail.com  Tue Jun 19 06:56:28 2012
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Tue, 19 Jun 2012 08:56:28 +0400
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
Message-ID: <CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>

As Amaury Forgeot d'Arc noted here,
https://bitbucket.org/pypy/pypy/pull-request/69/fix-issue-1180-implement-bool-formatting#comment-8635
in CPython there is no specific code for booleans, but an empty
format_spec always delegate to str().
Is it better to try to achieve the same in PyPy? Or just delegate to
str if there is no format string, and to int formatting if there is
any, like here https://bitbucket.org/kostialopuhin/pypy-fix1180/changeset/99ddf04ded5a
?

And a second questing - where is the proper place for this discussion
- here, in the pull request, or in the issues in bugtracker?

2012/6/19 Armin Rigo <arigo at tunes.org>:
> Hi,
>
> On Mon, Jun 18, 2012 at 10:54 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> It seems that you should make simpler tests (as
>> in there is no need for self.s and self.b I think, why not use str and bool
>> builtins?)
>
> Ah, it came from the other tests of test_newformat.py; but these use
> "self.s" instead of "str or unicode". ?"self.b()" is superflous, as
> you can use "bool()".
>
>
> A bient?t,
>
> Armin.

From amauryfa at gmail.com  Tue Jun 19 11:00:04 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 19 Jun 2012 11:00:04 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
	<CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
Message-ID: <CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>

2012/6/19 ????? ??????? <kostia.lopuhin at gmail.com>

> As Amaury Forgeot d'Arc noted here,
>
> https://bitbucket.org/pypy/pypy/pull-request/69/fix-issue-1180-implement-bool-formatting#comment-8635
> in CPython there is no specific code for booleans, but an empty
> format_spec always delegate to str().
> Is it better to try to achieve the same in PyPy? Or just delegate to
> str if there is no format string, and to int formatting if there is
> any, like here
> https://bitbucket.org/kostialopuhin/pypy-fix1180/changeset/99ddf04ded5a
> ?
>

Actually I think I found the cause:
bool.__format__ will correctly call format__Int_ANY(),
but the object is coerced into a int!

Adding a format__Bool_ANY fixes the problem (with the same implementation)1
but shouldn't we have W_BoolObject inherit from W_IntObject,
and remove delegate_Bool2IntObject?

And a second questing - where is the proper place for this discussion
> - here, in the pull request, or in the issues in bugtracker?
>

I don't think the volume of this mailing list is too high.
For quick round-trips, most of us also hang on IRC: #pypy on
irc.freenode.net.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/2e17a113/attachment.html>

From amauryfa at gmail.com  Tue Jun 19 13:30:53 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 19 Jun 2012 13:30:53 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
Message-ID: <CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>

2012/6/19 Leonard de HARO <leonard.de.haro at ens.fr>

> Hello,
>
> I'm a french undergraduated CS. student currently doing an internship in
> Santigo's University under the supervision of Eric Tanter. I'm working on
> the meta JIT compiler provided in the pypy project, trying to document
> it's performance on different kind of interpreters. I started two weeks
> ago.
>
> I have a problem with the translation toolchain. I've been writing an
> interpreter for a small language inspired by Shriram Krishnamurthi's F1WAE
> from his book "Programming Languages: Application and Interpretation".
> My problem is the following: I have an interpreter that seems to work
> perfecty (at least on my examples) when I use pypy interpreters, but
> whenever I translate it, execution fails. The fact is: I have identified
> the source of the problem but I don't understand it.
> If you look at the parser file, on line 184, there is a "print"
> instruction. With this instruction on comment, the translated version of
> RPinterpret executed on test10run10 gets a Stack Overflow. Use a
> translated version of the same file (RPinterpret.py) with the print
> instruction in parser.py, you get the normal behavior (which you have
> anyway using "pypy RPinterpret.py test10runs10").


> Looks like a bug, but since I'm a beginer (both on this work and on Python
> in general), maybe I have missed something.
>

Hum, looks like a RPython oddness.
It seems that you have to catch IndexError for IndexError to be raised by
the operation...

def CutWord(string, i):
    try:
        if string[i] in (' ', '\n', '\t'):
            return i
    except IndexError:
        return i
    return CutWord(string, i+1)

But I think a loop would be much more efficient...

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/ea2e2e68/attachment.html>

From wilk at flibuste.net  Tue Jun 19 15:51:11 2012
From: wilk at flibuste.net (wilk)
Date: Tue, 19 Jun 2012 13:51:11 +0000 (UTC)
Subject: [pypy-dev] set with instances very slow from 1.8 to 1.9
Message-ID: <jrq04f$68m$1@dough.gmane.org>

Hi,

I could isolate a piece of my code that show a big regression from 1.8 
to 1.9, event from cpython to 1.9 with operations on set of instances.  
On the other side set of integer are faster !

Should i open an issue on the bug tracker ?


$ python t.py
with instances = 0.02350
with integers =  0.02825

$ pypy-1.8 t.py
with instances = 0.00935
with integers =  0.02891

$ pypy-1.9 t.py
with instances = 0.41753
with integers =  0.00628

pypy-last/bin/pypy t.py
with instances = 0.34686
with integers =  0.00628


import random
import time
import sys

class X:
    pass

def bcl(a,b):
    for i in range(1000):
        a |= b

x1 = set([X() for i in range(1000)])
x2 = set([X() for i in range(1000)])

start = time.time()

bcl(x1, x2)

print "with instances = %.05f" % (time.time()-start)

x1 = set(range(1000))
x2 = set(range(1000))

start = time.time()

bcl(x1, x2)

print "with integers =  %.05f" % (time.time()-start)



-- 
William Dod?
Informaticien Ind?pendant


From Lukas.Diekmann at uni-duesseldorf.de  Tue Jun 19 18:33:07 2012
From: Lukas.Diekmann at uni-duesseldorf.de (Lukas Diekmann)
Date: Tue, 19 Jun 2012 18:33:07 +0200
Subject: [pypy-dev] set with instances very slow from 1.8 to 1.9
In-Reply-To: <jrq04f$68m$1@dough.gmane.org>
References: <jrq04f$68m$1@dough.gmane.org>
Message-ID: <4FE0A9C3.8080308@uni-duesseldorf.de>

Hi,

you are right. Union between two sets of objects are slower, since we
introduced mechanics to make set operations with primitive types (e.g.
ints) faster.
The problem was that a fastpath was removed in that process. I readded
it. Union/Update with sets of objects should be fast again in the future.

Thank you for figuring that out. Your program was very helpful.

Commit: https://bitbucket.org/pypy/pypy/changeset/452bb2805140

Lukas

Am 19.06.2012 15:51, schrieb wilk:
> Hi,
>
> I could isolate a piece of my code that show a big regression from 1.8 
> to 1.9, event from cpython to 1.9 with operations on set of instances.  
> On the other side set of integer are faster !
>
> Should i open an issue on the bug tracker ?
>
>
> $ python t.py
> with instances = 0.02350
> with integers =  0.02825
>
> $ pypy-1.8 t.py
> with instances = 0.00935
> with integers =  0.02891
>
> $ pypy-1.9 t.py
> with instances = 0.41753
> with integers =  0.00628
>
> pypy-last/bin/pypy t.py
> with instances = 0.34686
> with integers =  0.00628
>
>
> import random
> import time
> import sys
>
> class X:
>     pass
>
> def bcl(a,b):
>     for i in range(1000):
>         a |= b
>
> x1 = set([X() for i in range(1000)])
> x2 = set([X() for i in range(1000)])
>
> start = time.time()
>
> bcl(x1, x2)
>
> print "with instances = %.05f" % (time.time()-start)
>
> x1 = set(range(1000))
> x2 = set(range(1000))
>
> start = time.time()
>
> bcl(x1, x2)
>
> print "with integers =  %.05f" % (time.time()-start)
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120619/c8d6b1f4/attachment.html>

From dje.gcc at gmail.com  Tue Jun 19 19:47:52 2012
From: dje.gcc at gmail.com (David Edelsohn)
Date: Tue, 19 Jun 2012 13:47:52 -0400
Subject: [pypy-dev] set with instances very slow from 1.8 to 1.9
In-Reply-To: <4FE0A9C3.8080308@uni-duesseldorf.de>
References: <jrq04f$68m$1@dough.gmane.org>
	<4FE0A9C3.8080308@uni-duesseldorf.de>
Message-ID: <CAGWvny=NVZGh+UL8NTtB4iEtDQV1taUb1sN4BJ33d5jRybmycQ@mail.gmail.com>

On Tue, Jun 19, 2012 at 12:33 PM, Lukas Diekmann
<Lukas.Diekmann at uni-duesseldorf.de> wrote:
> Hi,
>
> you are right. Union between two sets of objects are slower, since we
> introduced mechanics to make set operations with primitive types (e.g. ints)
> faster.
> The problem was that a fastpath was removed in that process. I readded it.
> Union/Update with sets of objects should be fast again in the future.
>
> Thank you for figuring that out. Your program was very helpful.
>
> Commit: https://bitbucket.org/pypy/pypy/changeset/452bb2805140

Lukas,

Should the restored code include a comment about why the fast path is
useful for non-primitive types? Or a new test to ensure the optimal IR
is generated?

- David

From kostia.lopuhin at gmail.com  Tue Jun 19 21:18:19 2012
From: kostia.lopuhin at gmail.com (=?KOI8-R?B?68/T1NEg7M/Q1cjJzg==?=)
Date: Tue, 19 Jun 2012 23:18:19 +0400
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
	<CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
	<CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>
Message-ID: <CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>

2012/6/19 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
>
> Actually I think I found the cause:
> bool.__format__ will correctly call format__Int_ANY(),
> but the object is coerced into a int!
>
> Adding a format__Bool_ANY fixes the problem (with the same implementation)1
> but shouldn't we have W_BoolObject inherit from W_IntObject,
> and remove delegate_Bool2IntObject?
>

I tried inheriting W_BoolObject from W_IntObject, removing
delegate_Bool2IntObject and removing format__Bool_ANY - but then there
is no format implementation for bool. And as I read in multimethod.py,
it does not really interact with subclassing, so inheriting from
W_IntObject will gain nothing here?
Is it better to submit an easy fix first
https://bitbucket.org/kostialopuhin/pypy-fix1180/compare/..pypy/pypy ?
I just don't see what are the benefits of inhering from W_IntObject,
cause I know too little yet)

From wilk at flibuste.net  Wed Jun 20 11:49:22 2012
From: wilk at flibuste.net (wilk)
Date: Wed, 20 Jun 2012 09:49:22 +0000 (UTC)
Subject: [pypy-dev] set with instances very slow from 1.8 to 1.9
References: <jrq04f$68m$1@dough.gmane.org>
	<4FE0A9C3.8080308@uni-duesseldorf.de>
Message-ID: <jrs6b2$97c$1@dough.gmane.org>

On 19-06-2012, Lukas Diekmann wrote:

> Hi,
>
> you are right. Union between two sets of objects are slower, since we
> introduced mechanics to make set operations with primitive types (e.g.
> ints) faster.
> The problem was that a fastpath was removed in that process. I readded
> it. Union/Update with sets of objects should be fast again in the future.
>
> Thank you for figuring that out. Your program was very helpful.
>
> Commit: https://bitbucket.org/pypy/pypy/changeset/452bb2805140

Thanks, i can confirm, the last build is ok

-- 
William Dod?
Informaticien Ind?pendant


From fijall at gmail.com  Wed Jun 20 12:38:41 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 20 Jun 2012 12:38:41 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
	<CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
	<CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>
	<CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>
Message-ID: <CAK5idxQBBrRCDUKQdRYn_qf5YSa_pCVAORVRzBp4mq=RwRujAw@mail.gmail.com>

On Tue, Jun 19, 2012 at 9:18 PM, ????? ??????? <kostia.lopuhin at gmail.com>wrote:

> 2012/6/19 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> >
> > Actually I think I found the cause:
> > bool.__format__ will correctly call format__Int_ANY(),
> > but the object is coerced into a int!
> >
> > Adding a format__Bool_ANY fixes the problem (with the same
> implementation)1
> > but shouldn't we have W_BoolObject inherit from W_IntObject,
> > and remove delegate_Bool2IntObject?
> >
>
> I tried inheriting W_BoolObject from W_IntObject, removing
> delegate_Bool2IntObject and removing format__Bool_ANY - but then there
> is no format implementation for bool. And as I read in multimethod.py,
> it does not really interact with subclassing, so inheriting from
> W_IntObject will gain nothing here?
> Is it better to submit an easy fix first
> https://bitbucket.org/kostialopuhin/pypy-fix1180/compare/..pypy/pypy ?
> I just don't see what are the benefits of inhering from W_IntObject,
> cause I know too little yet)
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

bool inherits frmo int, just not W_BoolObject from W_IntObject. We should
and we should remove multimethods alltogether (they're a mess), but it's
quite a bit of effort.

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120620/0e8548e4/attachment.html>

From fijall at gmail.com  Wed Jun 20 12:39:17 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 20 Jun 2012 12:39:17 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAK5idxQBBrRCDUKQdRYn_qf5YSa_pCVAORVRzBp4mq=RwRujAw@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
	<CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
	<CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>
	<CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>
	<CAK5idxQBBrRCDUKQdRYn_qf5YSa_pCVAORVRzBp4mq=RwRujAw@mail.gmail.com>
Message-ID: <CAK5idxSqgnc1AVjEnwduWAcPvg1-wir=ssBEvPMFc586Vsa=Jg@mail.gmail.com>

On Wed, Jun 20, 2012 at 12:38 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Tue, Jun 19, 2012 at 9:18 PM, ????? ??????? <kostia.lopuhin at gmail.com>wrote:
>
>> 2012/6/19 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
>> >
>> > Actually I think I found the cause:
>> > bool.__format__ will correctly call format__Int_ANY(),
>> > but the object is coerced into a int!
>> >
>> > Adding a format__Bool_ANY fixes the problem (with the same
>> implementation)1
>> > but shouldn't we have W_BoolObject inherit from W_IntObject,
>> > and remove delegate_Bool2IntObject?
>> >
>>
>> I tried inheriting W_BoolObject from W_IntObject, removing
>> delegate_Bool2IntObject and removing format__Bool_ANY - but then there
>> is no format implementation for bool. And as I read in multimethod.py,
>> it does not really interact with subclassing, so inheriting from
>> W_IntObject will gain nothing here?
>> Is it better to submit an easy fix first
>> https://bitbucket.org/kostialopuhin/pypy-fix1180/compare/..pypy/pypy ?
>> I just don't see what are the benefits of inhering from W_IntObject,
>> cause I know too little yet)
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>>
>
> bool inherits frmo int, just not W_BoolObject from W_IntObject. We should
> and we should remove multimethods alltogether (they're a mess), but it's
> quite a bit of effort.
>
> Cheers,
> fijal
>

Ah and the reason why it would not work right now is because multimethods
don't quite follow the typical inheritance chain.

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120620/92d819f1/attachment.html>

From fijall at gmail.com  Wed Jun 20 12:41:21 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 20 Jun 2012 12:41:21 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
Message-ID: <CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>

On Tue, Jun 19, 2012 at 1:30 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2012/6/19 Leonard de HARO <leonard.de.haro at ens.fr>
>
>> Hello,
>>
>> I'm a french undergraduated CS. student currently doing an internship in
>> Santigo's University under the supervision of Eric Tanter. I'm working on
>> the meta JIT compiler provided in the pypy project, trying to document
>> it's performance on different kind of interpreters. I started two weeks
>> ago.
>>
>> I have a problem with the translation toolchain. I've been writing an
>> interpreter for a small language inspired by Shriram Krishnamurthi's F1WAE
>> from his book "Programming Languages: Application and Interpretation".
>> My problem is the following: I have an interpreter that seems to work
>> perfecty (at least on my examples) when I use pypy interpreters, but
>> whenever I translate it, execution fails. The fact is: I have identified
>> the source of the problem but I don't understand it.
>> If you look at the parser file, on line 184, there is a "print"
>> instruction. With this instruction on comment, the translated version of
>> RPinterpret executed on test10run10 gets a Stack Overflow. Use a
>> translated version of the same file (RPinterpret.py) with the print
>> instruction in parser.py, you get the normal behavior (which you have
>> anyway using "pypy RPinterpret.py test10runs10").
>
>
>> Looks like a bug, but since I'm a beginer (both on this work and on Python
>> in general), maybe I have missed something.
>>
>
> Hum, looks like a RPython oddness.
> It seems that you have to catch IndexError for IndexError to be raised by
> the operation...
>
> def CutWord(string, i):
>     try:
>         if string[i] in (' ', '\n', '\t'):
>             return i
>     except IndexError:
>         return i
>     return CutWord(string, i+1)
>
> But I think a loop would be much more efficient...
>
> --
> Amaury Forgeot d'Arc
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
Yes, it is an RPython oddity but I believe even a documented one!
http://doc.pypy.org/en/latest/coding-guide.html#rpython-definition look for
lists.

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120620/0319ffb3/attachment.html>

From Lukas.Diekmann at uni-duesseldorf.de  Wed Jun 20 12:19:00 2012
From: Lukas.Diekmann at uni-duesseldorf.de (Lukas Diekmann)
Date: Wed, 20 Jun 2012 12:19:00 +0200
Subject: [pypy-dev] set with instances very slow from 1.8 to 1.9
In-Reply-To: <CAGWvny=NVZGh+UL8NTtB4iEtDQV1taUb1sN4BJ33d5jRybmycQ@mail.gmail.com>
References: <jrq04f$68m$1@dough.gmane.org>
	<4FE0A9C3.8080308@uni-duesseldorf.de>
	<CAGWvny=NVZGh+UL8NTtB4iEtDQV1taUb1sN4BJ33d5jRybmycQ@mail.gmail.com>
Message-ID: <4FE1A394.5010402@uni-duesseldorf.de>

It is just a small optimization. Instead of manually copying all
elements from one set over to the other, the update method of rdict is
used. This optimization has already been there before I added strategies
for sets. I think it just got lost when I rewrote the W_SetObject
implementation. The same optimization is also used for the update method
of W_DictMultiObject. I guess the rdict is a little more optimized for
that task.

You are right. There wasn't a test checking this specific path (updating
an object set with another object set). So I added one.

- Lukas

Am 19.06.2012 19:47, schrieb David Edelsohn:
> On Tue, Jun 19, 2012 at 12:33 PM, Lukas Diekmann
> <Lukas.Diekmann at uni-duesseldorf.de> wrote:
>> Hi,
>>
>> you are right. Union between two sets of objects are slower, since we
>> introduced mechanics to make set operations with primitive types (e.g. ints)
>> faster.
>> The problem was that a fastpath was removed in that process. I readded it.
>> Union/Update with sets of objects should be fast again in the future.
>>
>> Thank you for figuring that out. Your program was very helpful.
>>
>> Commit: https://bitbucket.org/pypy/pypy/changeset/452bb2805140
> Lukas,
>
> Should the restored code include a comment about why the fast path is
> useful for non-primitive types? Or a new test to ensure the optimal IR
> is generated?
>
> - David

From arigo at tunes.org  Wed Jun 20 17:46:55 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 20 Jun 2012 17:46:55 +0200
Subject: [pypy-dev] A very easy bug for newcomers
In-Reply-To: <CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>
References: <CAK5idxSei6pYyk+uu-uRkSEvqLrGxXa=dP3d6s2k0Ts3Zgb+7g@mail.gmail.com>
	<CAAvCB=RRGASQ9GBA4XFR1dvzA5FkbbrZR59Xkuhkfsd8OBk8CA@mail.gmail.com>
	<CAMSv6X0=0xVoZaRWux+aZrd8rSzPib6mZ8fZQ2xK64G0OjkHng@mail.gmail.com>
	<CAAvCB=S+P7xqu8UGgzF1aytkp1bJiqeTyULTEiH7xQcajW94=Q@mail.gmail.com>
	<CAAvCB=QLU5urBD7C0+kXpG8XyPSCm=+k65hfj5p=UP++yAB_7A@mail.gmail.com>
	<CAK5idxTpAifBbXMBjX8h8P7uFUGFDKy12X3x15y6pvc-c_iQxg@mail.gmail.com>
	<CAMSv6X20FFjtDgtOsQLdomwd9y=hUhfN=TO10jZe102mXPx8pg@mail.gmail.com>
	<CAAvCB=TdTwgNNv5Wze772JpyEX+vFR6TuFHVPL5DzCnDkzTTQg@mail.gmail.com>
	<CAGmFidZ9Ztz2H4D=VjewV5+WV=jGJeciD3AF4pWyUMW9=JH2hQ@mail.gmail.com>
	<CAAvCB=Sh1Lz_iugedJ8-yFc3feU_Yiq2zxL9vP7ove_G2qTWnQ@mail.gmail.com>
Message-ID: <CAMSv6X18Jspkw4NY3LC+2Gt+hOtoX=+XX9ygDK2p9Pq5dDuZCw@mail.gmail.com>

Hi,

On Tue, Jun 19, 2012 at 9:18 PM, ????? ??????? <kostia.lopuhin at gmail.com> wrote:
> Is it better to submit an easy fix first
> https://bitbucket.org/kostialopuhin/pypy-fix1180/compare/..pypy/pypy ?

Yes, I suppose that we should commit the easy fix.


Armin

From leonard.de.haro at ens.fr  Wed Jun 20 18:23:49 2012
From: leonard.de.haro at ens.fr (=?UTF-8?B?TMOpb25hcmQgZGUgSGFybw==?=)
Date: Wed, 20 Jun 2012 12:23:49 -0400
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
	<CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
Message-ID: <4FE1F915.1080805@ens.fr>

Thanks for the answer. I had read the doc but didn't fully understood it 
until I experienced it!

I have a few more questions if you don't mind. These should be less 
trivial than the previous one.

I'm trying to experience the efficiency of the translation and jit-ing 
tools provided by pypy on AST-based interpreters at different level of 
complexity (from completly recursive toward completly iterative).

To test the speed of the interpreters, I use a program that write files: 
given n and runs it produces two functions and a call. The first 
function is f : x -> n*x but using a tree and only the + operator. The 
second one is recursive
g : x -> if x=0 (f 3) else (f 3), (g (x-1)). Then I add a call: (g runs).
The aim is to force the JITing versions to recognize the instructions f 
3 as needed to be traced and do it.

I have the following problems:

When translating the two JITing interpreters that I have (one purely 
recursive, the second one tail-recursive in CPS), I get this warning:

[platform:WARNING] rpython_lltypesystem_rffi.c: In function 
?pypy_g__PyPy_dg_dtoa__Float_Signed_Signed_arrayPtr_arra?:
[platform:WARNING] rpython_lltypesystem_rffi.c:534: warning: passing 
argument 4 of ?_PyPy_dg_dtoa? from incompatible pointer type
[platform:WARNING] rpython_lltypesystem_rffi.c:534: warning: passing 
argument 5 of ?_PyPy_dg_dtoa? from incompatible pointer type

Whenever the tests take longer than a couple of seconds to be executed 
by the non-JITing interpreter, JITing version get the following stack 
overflow:

[...]
   File "jit_metainterp_warmspot.c", line 194, in 
ll_portal_runner__treeClass_F1WAEPtr_dicttablePt
   File "implement.c", line 11681, in portal
   File "jit_metainterp_warmspot.c", line 194, in 
ll_portal_runner__treeClass_F1WAEPtr_dicttablePt
   File "implement.c", line 12345, in portal
   File "jit_metainterp_warmspot.c", line 194, in 
ll_portal_runner__treeClass_F1WAEPtr_dicttablePt
   File "implement.c", line 12345, in portal
   ...
Fatal RPython error: StackOverflow
Abort trap


The non JIT-ing version passes the test (n=10000, runs = 2000) in 2.4s, 
the JITing counter-part takes 45s to raise the error when the cps JITing 
version only takes 0.3s to stop.

I had already noticed the huge difference between JITing and non-JITing 
versions (purely recursive) on tests that declare functions (they run in 
similar time whithout function declaration). I guess it is due to the 
"should I trace" phase.

My interpreting function takes an AST, a dictionnary of function 
declaration and an environment (cps version has the continuation as 
well). Everything is declared green.

You can find these files here: 
https://github.com/zebign1/RPython-internship/tree/master/Interpreters/ifF1WAE

(I just came accross your tutorial on bitbucket, so I didn't know there 
were parsing tools available with RPython ? when I try to import the 
parser module, translation failed ? so I wrote my own parser, whici is 
probably less efficient, but all the interpreters share the same.)

Here are my questions:

- What is the warning?
- What is the stack overflow? (I first thought it was the same bug as 
before but it's not within a function I've written apparently)
- Is it normal that the JITing version takes 35x longer than the non 
JITing version (is it due to the recursive form ?)?
- Have I made any mistakes declaring all my variables green? (it is 
highly possible that I missed something here)
- Your tutorial transformed your AST into bytecode. Why?

Again, thank you for the answer.

Cheers
Leonard.

From amauryfa at gmail.com  Wed Jun 20 19:02:55 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 20 Jun 2012 19:02:55 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <4FE1F915.1080805@ens.fr>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
	<CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
	<4FE1F915.1080805@ens.fr>
Message-ID: <CAGmFidb1+B2iWhGaC_HjkkERdTQs9K+w1ipmGzTXfVX-X-r=1w@mail.gmail.com>

2012/6/20 L?onard de Haro <leonard.de.haro at ens.fr>

> When translating the two JITing interpreters that I have (one purely
> recursive, the second one tail-recursive in CPS), I get this warning:
>
> [platform:WARNING] rpython_lltypesystem_rffi.c: In function
> ?pypy_g__PyPy_dg_dtoa__Float_**Signed_Signed_arrayPtr_arra?:
> [platform:WARNING] rpython_lltypesystem_rffi.c:**534: warning: passing
> argument 4 of ?_PyPy_dg_dtoa? from incompatible pointer type
> [platform:WARNING] rpython_lltypesystem_rffi.c:**534: warning: passing
> argument 5 of ?_PyPy_dg_dtoa? from incompatible pointer type
>

This is mostly harmless. It happens because the C function _PyPy_dg_dtoa is
declared with int* parameters,
and generated code calls it with long* values.
But this only happens when int and long are of the same size (= on 32bit
platforms),
so nothing bad can happen here.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120620/b2d6797c/attachment.html>

From csparr at ucsd.edu  Wed Jun 20 21:40:29 2012
From: csparr at ucsd.edu (Cameron Sparr)
Date: Wed, 20 Jun 2012 13:40:29 -0600
Subject: [pypy-dev] __format__ support for float64 type in NumPyPy
Message-ID: <4FE2272D.3080403@ucsd.edu>

Hi all,

I noticed that there is no support for the .__format__() function with 
float64 data types in NumPyPy... I'm pretty new and was wondering if 
this is something that I should submit a bug report for?

Cameron


-- 
Cameron Sparr
SIParCS Intern
NCAR \ CISL \ User Services Section (USS)
phone:  303-497-2443
mobile: 503-327-5964
email:  csparr at ucar.edu


From p.j.a.cock at googlemail.com  Thu Jun 21 00:24:32 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Wed, 20 Jun 2012 23:24:32 +0100
Subject: [pypy-dev] PyPy 1.9 on LaunchPad PPA?
Message-ID: <CAKVJ-_6h+PsrhvgvHYmqELZRz10zgRqt7LeKKvkQup0q_2hZ9g@mail.gmail.com>

Hello all,

I'd like to be able to run tests for a Python project on
Travis-CI under PyPy 1.9, however to provide PyPy
on their virtual machines Travis-CI get PyPy from the
PyPy PPA on LaunchPad:

http://about.travis-ci.org/docs/user/languages/python/
https://launchpad.net/~pypy/+archive/ppa

It appears that currently the latest release on the PPA is
still PyPy 1.8 (and sadly for the project I have in mind,
I require PyPy 1.9).

Could someone update the PPA please?

Thanks,

Peter

From anto.cuni at gmail.com  Thu Jun 21 09:49:25 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 21 Jun 2012 09:49:25 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <4FE1F915.1080805@ens.fr>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
	<CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
	<4FE1F915.1080805@ens.fr>
Message-ID: <4FE2D205.6000608@gmail.com>

Hi L?onard,

On 06/20/2012 06:23 PM, L?onard de Haro wrote:
> 
> My interpreting function takes an AST, a dictionnary of function declaration
> and an environment (cps version has the continuation as well). Everything is
> declared green.

I didn't read the code, but this looks wrong. You should declare the variables
representing "the program" as green, and the variables representing "the
state" as red. Else, the JIT tries to specialize *everything* and you never
get fast code.

This might also be the cause of your stack overflow (blind guess). Try to
declare the environment as red (and maybe the dictionary of functions, it
depends whether it is static or dynamic).

ciao,
Anto

From fijall at gmail.com  Thu Jun 21 12:25:34 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 21 Jun 2012 12:25:34 +0200
Subject: [pypy-dev] __format__ support for float64 type in NumPyPy
In-Reply-To: <4FE2272D.3080403@ucsd.edu>
References: <4FE2272D.3080403@ucsd.edu>
Message-ID: <CAK5idxT64P1OTRsJM2ViyrG2Xfp-2oOgLzPfAheZsT00NO_Egg@mail.gmail.com>

On Wed, Jun 20, 2012 at 9:40 PM, Cameron Sparr <csparr at ucsd.edu> wrote:

> Hi all,
>
> I noticed that there is no support for the .__format__() function with
> float64 data types in NumPyPy... I'm pretty new and was wondering if this
> is something that I should submit a bug report for?
>
> Cameron
>

I would like to avoid having a ticket-per-missing-feature, since that would
be lots of tickets. We know the numpy support is not done by far.

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120621/7c67e435/attachment.html>

From fijall at gmail.com  Thu Jun 21 12:26:40 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 21 Jun 2012 12:26:40 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <4FE2D205.6000608@gmail.com>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
	<CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
	<4FE1F915.1080805@ens.fr> <4FE2D205.6000608@gmail.com>
Message-ID: <CAK5idxS1nwwG68Nu5U+h_DYrJOhoXn4v6Fjstwv_Ok1ndOwQ4w@mail.gmail.com>

On Thu, Jun 21, 2012 at 9:49 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> Hi L?onard,
>
> On 06/20/2012 06:23 PM, L?onard de Haro wrote:
> >
> > My interpreting function takes an AST, a dictionnary of function
> declaration
> > and an environment (cps version has the continuation as well).
> Everything is
> > declared green.
>
> I didn't read the code, but this looks wrong. You should declare the
> variables
> representing "the program" as green, and the variables representing "the
> state" as red. Else, the JIT tries to specialize *everything* and you never
> get fast code.
>

You get very fast code, but you can never use it :) It's used precisely once
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120621/9c3943fd/attachment.html>

From arigo at tunes.org  Thu Jun 21 12:50:42 2012
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 21 Jun 2012 12:50:42 +0200
Subject: [pypy-dev] Internship and bug report
In-Reply-To: <4FE1F915.1080805@ens.fr>
References: <c065595641a24cf07d6d95a44d98b086.squirrel@squirrelmail.eleves.ens.fr>
	<CAGmFidaDiXDvxYii8VXZP2XAPfdj5TvUa6Xo6YN-u3weXvS82Q@mail.gmail.com>
	<CAK5idxSh36BiodZUmueqVtcO0KQ2=5TUro5Mpy+dqi1xr+HMsw@mail.gmail.com>
	<4FE1F915.1080805@ens.fr>
Message-ID: <CAMSv6X0Lp=kWBgsOYH-ayZFM4zgijdfv_Jzh4nbEJu+SBWTdKQ@mail.gmail.com>

Hi L?onard,

On Wed, Jun 20, 2012 at 6:23 PM, L?onard de Haro <leonard.de.haro at ens.fr> wrote:
> - What is the stack overflow?

It's just what it says: a stack overflow.  RPython has no CPS or
tail-recursion support, so it doesn't support arbitrary heap-limited
recursion.  Have a look at the Prolog interpreter
(https://bitbucket.org/cfbolz/pyrolog) to see an example of how to
work around this.  It is also described in papers like "Towards a
jitting VM for prolog execution" or "Towards Just-In-Time partial
evaluation of prolog" (C.F. Bolz et al.).


A bient?t,

Armin.

From skip at pobox.com  Thu Jun 21 14:18:07 2012
From: skip at pobox.com (Skip Montanaro)
Date: Thu, 21 Jun 2012 07:18:07 -0500
Subject: [pypy-dev] __format__ support for float64 type in NumPyPy
In-Reply-To: <CAK5idxT64P1OTRsJM2ViyrG2Xfp-2oOgLzPfAheZsT00NO_Egg@mail.gmail.com>
References: <4FE2272D.3080403@ucsd.edu>
	<CAK5idxT64P1OTRsJM2ViyrG2Xfp-2oOgLzPfAheZsT00NO_Egg@mail.gmail.com>
Message-ID: <CANc-5Uw4ssb=wo8_VdFsP7PN0F98_Y-sK99-h042Wo0+t+J3MA@mail.gmail.com>

Maybe a single "missing features" ticket for numpypy to which  people can
add and prioritize would make sense?

Skip
On Jun 21, 2012 5:27 AM, "Maciej Fijalkowski" <fijall at gmail.com> wrote:

> On Wed, Jun 20, 2012 at 9:40 PM, Cameron Sparr <csparr at ucsd.edu> wrote:
>
>> Hi all,
>>
>> I noticed that there is no support for the .__format__() function with
>> float64 data types in NumPyPy... I'm pretty new and was wondering if this
>> is something that I should submit a bug report for?
>>
>> Cameron
>>
>
> I would like to avoid having a ticket-per-missing-feature, since that
> would be lots of tickets. We know the numpy support is not done by far.
>
> Cheers,
> fijal
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120621/24f1c370/attachment.html>

From naylor.b.david at gmail.com  Fri Jun 22 11:31:00 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Fri, 22 Jun 2012 11:31:00 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAMSv6X2fNtPox79nHRK0H3JrU_jt3D4DR-177Hf4-wbdzRspAA@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206161641.59677.naylor.b.david@gmail.com>
	<CAMSv6X2fNtPox79nHRK0H3JrU_jt3D4DR-177Hf4-wbdzRspAA@mail.gmail.com>
Message-ID: <201206221131.07207.naylor.b.david@gmail.com>

On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:
> Hi David,
> 
> On Sat, Jun 16, 2012 at 4:41 PM, David Naylor <naylor.b.david at gmail.com> wrote:
> > Oh, okay.  I don't have access to the latest sources.  I normally work of
> > the released sources.  But thanks for looking at this.
> 
> It's going to take us a loooooong time to fix CLI translation if the
> process goes: "you find some minor bug in the release, we attempt to
> fix it a bit blindly, and then we wait for the next release to know if
> it worked and what the next minor bug is" :-)

This is true.  Would it be possible to do a RC release, or a heads up, so that I may track down these small minor bugs *before* the release is 
made?  

> In case you want to help slightly more actively, you need to get the
> sources via Mercurial's command "hg clone
> https://bitbucket.org/pypy/pypy" and then keep up-to-date with "hg
> pull && hg update".  

Here are my results:
# mono --version
Mono JIT compiler version 2.11.1 (tarball Wed Jun 13 11:35:28 UTC 2012)
Copyright (C) 2002-2012 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           normal
        SIGSEGV:       normal
        Notification:  kqueue
        Architecture:  x86
        Disabled:      none
        Misc:          softdebug 
        LLVM:          supported, not enabled.
        GC:            Included Boehm (with typed GC and Parallel Mark)

1) First attempt (fixed with quick patch):
# python translate.py --source -O2 --backend=cli
<snip/>
[translation:ERROR]  Skipped: ilasm2 is not on your path.

2) Second attempt (fixed with quick patch):
# python translate.py --source -O2 --backend=cli
<snip/>
[translation:ERROR]  CompilationError': CompilationError(err="""
[translation:ERROR]     /tmp/usession-unknown-1/platcheck_4.c:38:17: error: pty.h: No such file or directory
[translation:ERROR]     """)
[translation:ERROR]     .. v1 = simple_call((builtin_function_or_method write), (2), v0)
[translation:ERROR]     .. '(targetpypystandalone:19)debug'
[translation:ERROR] Processing block:
[translation:ERROR]  block at 15 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (targetpypystandalone:19)debug
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v2 = add(('debug: '), msg_0)
[translation:ERROR]        v0 = add(v2, ('\n'))
[translation:ERROR]        v1 = simple_call((builtin_function_or_method write), (2), v0)
[translation:ERROR]  --end--

3) Third attempt:
# python translate.py --source -O2 --backend=cli
<snip/>
[rtyper] -=- specialized 10 more blocks -=-
[Timer] Timings:
[Timer] annotate                       --- 280.7 s
[Timer] rtype_ootype                   --- 367.2 s
[Timer] backendopt_ootype              ---  54.7 s
[Timer] source_cli                     ---   9.8 s
[Timer] ==========================================
[Timer] Total:                         --- 712.3 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 308, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/driver.py", line 791, in proceed
[translation:ERROR]     return self._execute(goals, task_skip = self._maybe_skip())
[translation:ERROR]    File "/tmp/pypy/pypy/translator/tool/taskengine.py", line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/driver.py", line 285, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File "/tmp/pypy/pypy/translator/driver.py", line 618, in task_source_cli
[translation:ERROR]     filename = self.gen.generate_source()
[translation:ERROR]    File "/tmp/pypy/pypy/translator/cli/gencli.py", line 56, in generate_source
[translation:ERROR]     GenOO.generate_source(self)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/genoo.py", line 68, in generate_source
[translation:ERROR]     self.gen_pendings()
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/genoo.py", line 83, in gen_pendings
[translation:ERROR]     node.render(self.ilasm)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/function.py", line 131, in render
[translation:ERROR]     self.render_normal_block(block)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/function.py", line 239, in render_normal_block
[translation:ERROR]     self._render_op(op)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/function.py", line 377, in _render_op
[translation:ERROR]     instr_list.render(self.generator, op)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 246, in render
[translation:ERROR]     instr.render(generator, op)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/cli/metavm.py", line 27, in render
[translation:ERROR]     _OOCall.render(self, generator, op)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 456, in render
[translation:ERROR]     is_primitive = get_primitive_name(callee)
[translation:ERROR]    File "/tmp/pypy/pypy/translator/oosupport/metavm.py", line 450, in get_primitive_name
[translation:ERROR]     return sm._name.rsplit('.', 1)
[translation:ERROR]    File "/tmp/pypy/pypy/rpython/lltypesystem/lltype.py", line 1171, in __getattr__
[translation:ERROR]     field_name))
[translation:ERROR]  AttributeError: <Func (  ) -> Signed> instance has no field '_name'

To make sure things were working properly on my system I did:
# python translate.py --source -O2
with success.

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120622/6284c4d5/attachment-0001.pgp>

From naylor.b.david at gmail.com  Fri Jun 22 13:05:26 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Fri, 22 Jun 2012 13:05:26 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
Message-ID: <201206221305.30243.naylor.b.david@gmail.com>

Hi,

I'm experimenting with using non-default library directories by tweeking pypy.tool.lib_pypy however I came up with the following error 
when translating with pypy:

[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 274, in main
[translation:ERROR]     default_goal='compile')
[translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 810, in 
from_targetspec
[translation:ERROR]     spec = target(driver, args)
[translation:ERROR]    File "targetpypystandalone.py", line 228, in target
[translation:ERROR]     return self.get_entry_point(config)
[translation:ERROR]    File "targetpypystandalone.py", line 236, in get_entry_point
[translation:ERROR]     rebuild = import_from_lib_pypy('ctypes_config_cache/rebuild')
[translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/tool/lib_pypy.py", line 12, in 
import_from_lib_pypy
[translation:ERROR]     return modname.pyimport()
[translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/py/_path/local.py", line 531, in pyimport
[translation:ERROR]     mod = __import__(modname, None, None, ['__doc__'])
[translation:ERROR]    File "/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/rebuild.py", line 11, in <module>
[translation:ERROR]     execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
[translation:ERROR]  IOError: [Errno 2] No such file or directory: '/usr/local/pypy-1.9/pypy/tool/autopath.py'

This error does not happen when translating with python.

Upon investigation I found that pypy has:
# pypy -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()'
True

but python has:
# python -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()'
False

also:
# pypy -c 'import sys; import pypy.tool.lib_pypy; print sys.modules["ctypes_config_cache"]'
<module 'ctypes_config_cache' from '/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/__init__.pyc'>
note the above always happens regardless of pypy.tool.lib_pypy being modified or not.  

It appears to me that there are two bugs:
1) pypy ends up importing ctypes_config_cache when it shouldn't???
2) py/_path/local.py doesn't ensure import from correct location

And attached is a patch for fixing (2), which works around (1).  

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch-py___path__local.py
Type: text/x-python
Size: 593 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120622/d93c35ee/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120622/d93c35ee/attachment.pgp>

From stefano at rivera.za.net  Fri Jun 22 15:09:39 2012
From: stefano at rivera.za.net (Stefano Rivera)
Date: Fri, 22 Jun 2012 15:09:39 +0200
Subject: [pypy-dev] PyPy 1.9 on LaunchPad PPA?
In-Reply-To: <CAKVJ-_6h+PsrhvgvHYmqELZRz10zgRqt7LeKKvkQup0q_2hZ9g@mail.gmail.com>
References: <CAKVJ-_6h+PsrhvgvHYmqELZRz10zgRqt7LeKKvkQup0q_2hZ9g@mail.gmail.com>
Message-ID: <20120622130939.GT1728@bach.rivera.co.za>

Hi Peter (2012.06.21_00:24:32_+0200)
> I'd like to be able to run tests for a Python project on
> Travis-CI under PyPy 1.9, however to provide PyPy
> on their virtual machines Travis-CI get PyPy from the
> PyPy PPA on LaunchPad:
> 
> http://about.travis-ci.org/docs/user/languages/python/
> https://launchpad.net/~pypy/+archive/ppa
> 
> It appears that currently the latest release on the PPA is
> still PyPy 1.8 (and sadly for the project I have in mind,
> I require PyPy 1.9).
> 
> Could someone update the PPA please?

Published builds for precise this afternoon.
Oneiric is still in progress.

Are you needing them for a different release? (They will be harder to
prepare)

SR

-- 
Stefano Rivera
  http://tumbleweed.org.za/
  H: +27 21 461 1230 C: +27 72 419 8559

From fijall at gmail.com  Fri Jun 22 15:29:41 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 22 Jun 2012 15:29:41 +0200
Subject: [pypy-dev] PyPy 1.9 on LaunchPad PPA?
In-Reply-To: <20120622130939.GT1728@bach.rivera.co.za>
References: <CAKVJ-_6h+PsrhvgvHYmqELZRz10zgRqt7LeKKvkQup0q_2hZ9g@mail.gmail.com>
	<20120622130939.GT1728@bach.rivera.co.za>
Message-ID: <CAK5idxQ=uZ+O0rNC3SVpij8aU58OLQzpSVRVRTcmhY=9G6Ktkg@mail.gmail.com>

On Fri, Jun 22, 2012 at 3:09 PM, Stefano Rivera <stefano at rivera.za.net>wrote:

> Hi Peter (2012.06.21_00:24:32_+0200)
> > I'd like to be able to run tests for a Python project on
> > Travis-CI under PyPy 1.9, however to provide PyPy
> > on their virtual machines Travis-CI get PyPy from the
> > PyPy PPA on LaunchPad:
> >
> > http://about.travis-ci.org/docs/user/languages/python/
> > https://launchpad.net/~pypy/+archive/ppa
> >
> > It appears that currently the latest release on the PPA is
> > still PyPy 1.8 (and sadly for the project I have in mind,
> > I require PyPy 1.9).
> >
> > Could someone update the PPA please?
>
> Published builds for precise this afternoon.
> Oneiric is still in progress.
>
> Are you needing them for a different release? (They will be harder to
> prepare)
>
> SR
>
> --
> Stefano Rivera
>  http://tumbleweed.org.za/
>  H: +27 21 461 1230 C: +27 72 419 8559
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

Thanks stefano!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120622/c768ea1b/attachment.html>

From dje.gcc at gmail.com  Fri Jun 22 15:41:04 2012
From: dje.gcc at gmail.com (David Edelsohn)
Date: Fri, 22 Jun 2012 09:41:04 -0400
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <201206221131.07207.naylor.b.david@gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206161641.59677.naylor.b.david@gmail.com>
	<CAMSv6X2fNtPox79nHRK0H3JrU_jt3D4DR-177Hf4-wbdzRspAA@mail.gmail.com>
	<201206221131.07207.naylor.b.david@gmail.com>
Message-ID: <CAGWvnykfC0gT4yBgQfFGGp9B4fe9fABuCtt4CDiSM86ESYedKQ@mail.gmail.com>

On Fri, Jun 22, 2012 at 5:31 AM, David Naylor <naylor.b.david at gmail.com> wrote:
> On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:

>> It's going to take us a loooooong time to fix CLI translation if the
>> process goes: "you find some minor bug in the release, we attempt to
>> fix it a bit blindly, and then we wait for the next release to know if
>> it worked and what the next minor bug is" :-)
>
> This is true. ?Would it be possible to do a RC release, or a heads up, so that I may track down these small minor bugs *before* the release is
> made?

You said that you do not have access to the latest sources, but never
explained why.  You also did not respond to Armin's question about
accessing source from the Bitbucket repository, or downloading source
tarballs or the periodic builds from the buildbot (buildbot.pypy.org).

If you want to help fix the CLI backend, why can't you experiment with
the development version of PyPy?

Thanks, David

From p.j.a.cock at googlemail.com  Fri Jun 22 16:07:25 2012
From: p.j.a.cock at googlemail.com (Peter Cock)
Date: Fri, 22 Jun 2012 15:07:25 +0100
Subject: [pypy-dev] PyPy 1.9 on LaunchPad PPA?
In-Reply-To: <20120622130939.GT1728@bach.rivera.co.za>
References: <CAKVJ-_6h+PsrhvgvHYmqELZRz10zgRqt7LeKKvkQup0q_2hZ9g@mail.gmail.com>
	<20120622130939.GT1728@bach.rivera.co.za>
Message-ID: <CAKVJ-_7gp-ouMDUvGk07CYJNLtpO+zhEhw4L4at0bcgULm-oyQ@mail.gmail.com>

On Fri, Jun 22, 2012 at 2:09 PM, Stefano Rivera <stefano at rivera.za.net> wrote:
> Hi Peter (2012.06.21_00:24:32_+0200)
>> I'd like to be able to run tests for a Python project on
>> Travis-CI under PyPy 1.9, however to provide PyPy
>> on their virtual machines Travis-CI get PyPy from the
>> PyPy PPA on LaunchPad:
>>
>> http://about.travis-ci.org/docs/user/languages/python/
>> https://launchpad.net/~pypy/+archive/ppa
>>
>> It appears that currently the latest release on the PPA is
>> still PyPy 1.8 (and sadly for the project I have in mind,
>> I require PyPy 1.9).
>>
>> Could someone update the PPA please?
>
> Published builds for precise this afternoon.
> Oneiric is still in progress.
>
> Are you needing them for a different release? (They will be
> harder to prepare)
>
> SR

Thanks.

I'm not entirely sure which Travis-CI is using, but I just tried
it and it was still using PyPy 1.8. That could mean a stale
cache somewhere, or more likely they are not using 'precise'.

I think I can workaround this by downloading and installing
the pre-compiled PyPy 1.9 directly - fingers crossed.

Peter

From naylor.b.david at gmail.com  Fri Jun 22 16:26:05 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Fri, 22 Jun 2012 16:26:05 +0200
Subject: [pypy-dev] CLI translate error with pypy-1.9
In-Reply-To: <CAGWvnykfC0gT4yBgQfFGGp9B4fe9fABuCtt4CDiSM86ESYedKQ@mail.gmail.com>
References: <201206121158.11984.naylor.b.david@gmail.com>
	<201206221131.07207.naylor.b.david@gmail.com>
	<CAGWvnykfC0gT4yBgQfFGGp9B4fe9fABuCtt4CDiSM86ESYedKQ@mail.gmail.com>
Message-ID: <201206221626.08884.naylor.b.david@gmail.com>

On Friday, 22 June 2012 15:41:04 David Edelsohn wrote:
> On Fri, Jun 22, 2012 at 5:31 AM, David Naylor <naylor.b.david at gmail.com> 
wrote:
> > On Sunday, 17 June 2012 09:32:35 Armin Rigo wrote:
> >> It's going to take us a loooooong time to fix CLI translation if the
> >> process goes: "you find some minor bug in the release, we attempt to
> >> fix it a bit blindly, and then we wait for the next release to know if
> >> it worked and what the next minor bug is" :-)
> > 
> > This is true.  Would it be possible to do a RC release, or a heads up, so
> > that I may track down these small minor bugs *before* the release is
> > made?
> 
> You said that you do not have access to the latest sources, but never
> explained why.  

It was due to mercurial not being installed on my computer, and only recently 
have I been able to install more software (hence my 5 day delayed reply to 
Armin's email).

> You also did not respond to Armin's question about
> accessing source from the Bitbucket repository, or downloading source
> tarballs or the periodic builds from the buildbot (buildbot.pypy.org).

My original email was based on the downloaded source for pypy-1.9, the follow 
up email to Armin was based on the bitbucket repository.  I don't know how to 
work mercurial very well but I think `hg log` shows the latest commit was 
55751:906bdd8dc432 made "Thu Jun 21 18:22:48 2012 +0200".  

> If you want to help fix the CLI backend, why can't you experiment with
> the development version of PyPy?

I have tried the developer version (on the main branch?) and reported back on 
my results.  

I am happy to help pypy, and I am now in a position to do so (on the developer 
version), however I would like to note that I have no need for the CLI backend 
and would prefer to not divert energy away from other areas of the project.  
But, to reiterate, I am happy, willing and able, to help where I can.  

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120622/848f4950/attachment-0001.pgp>

From arigo at tunes.org  Sun Jun 24 10:05:59 2012
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 24 Jun 2012 10:05:59 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <201206221305.30243.naylor.b.david@gmail.com>
References: <201206221305.30243.naylor.b.david@gmail.com>
Message-ID: <CAMSv6X2ocp0R=Eim-ZJkkkXep3qa+kagXmG7oG40VxFeKuwt5A@mail.gmail.com>

Hi David,

On Fri, Jun 22, 2012 at 1:05 PM, David Naylor <naylor.b.david at gmail.com> wrote:
> And attached is a patch for fixing (2), which works around (1).

Your patch modifies the py lib, more precisely py/_path.  The py lib
is nowadays a separate project.  You should try to discuss it and
submit it to the py lib mailing list.


Armin

From Ronny.Pfannschmidt at gmx.de  Sun Jun 24 10:31:33 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Sun, 24 Jun 2012 10:31:33 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <201206221305.30243.naylor.b.david@gmail.com>
References: <201206221305.30243.naylor.b.david@gmail.com>
Message-ID: <4FE6D065.7030605@gmx.de>

Hi David,

your change to pylib introduces a worse bug thats not acceptable

it really should error out in that case,
since it would enforce a partial code reload of the old code or even a 
wrong code reload for subpackages

-- Ronny


On 06/22/2012 01:05 PM, David Naylor wrote:
> Hi,
>
> I'm experimenting with using non-default library directories by tweeking pypy.tool.lib_pypy however I came up with the following error
> when translating with pypy:
>
> [translation:ERROR] Error:
> [translation:ERROR]  Traceback (most recent call last):
> [translation:ERROR]    File "translate.py", line 274, in main
> [translation:ERROR]     default_goal='compile')
> [translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 810, in
> from_targetspec
> [translation:ERROR]     spec = target(driver, args)
> [translation:ERROR]    File "targetpypystandalone.py", line 228, in target
> [translation:ERROR]     return self.get_entry_point(config)
> [translation:ERROR]    File "targetpypystandalone.py", line 236, in get_entry_point
> [translation:ERROR]     rebuild = import_from_lib_pypy('ctypes_config_cache/rebuild')
> [translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/tool/lib_pypy.py", line 12, in
> import_from_lib_pypy
> [translation:ERROR]     return modname.pyimport()
> [translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/py/_path/local.py", line 531, in pyimport
> [translation:ERROR]     mod = __import__(modname, None, None, ['__doc__'])
> [translation:ERROR]    File "/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/rebuild.py", line 11, in<module>
> [translation:ERROR]     execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
> [translation:ERROR]  IOError: [Errno 2] No such file or directory: '/usr/local/pypy-1.9/pypy/tool/autopath.py'
>
> This error does not happen when translating with python.
>
> Upon investigation I found that pypy has:
> # pypy -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()'
> True
>
> but python has:
> # python -c 'import sys; import pypy.tool.lib_pypy; print "ctypes_config_cache" in sys.modules.keys()'
> False
>
> also:
> # pypy -c 'import sys; import pypy.tool.lib_pypy; print sys.modules["ctypes_config_cache"]'
> <module 'ctypes_config_cache' from '/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/__init__.pyc'>
> note the above always happens regardless of pypy.tool.lib_pypy being modified or not.
>
> It appears to me that there are two bugs:
> 1) pypy ends up importing ctypes_config_cache when it shouldn't???
> 2) py/_path/local.py doesn't ensure import from correct location
>
> And attached is a patch for fixing (2), which works around (1).
>
> Regards
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From leonard.de.haro at ens.fr  Mon Jun 25 16:40:02 2012
From: leonard.de.haro at ens.fr (=?ISO-8859-1?Q?L=E9onard_de_Haro?=)
Date: Mon, 25 Jun 2012 10:40:02 -0400
Subject: [pypy-dev]  Help (again)
Message-ID: <4FE87842.3070209@ens.fr>

Hi everyone!

First of all, thaks for your answers last time. Second of all, I need 
your help again.

I'm coding an imperative version of my interpret and to do so, I coded 
continuations. I used a similat technique to the one used in the Prolog 
interpret you advised me to take a look at last time.

I define a serie of class all inheriting from a simulated-abstract Contk 
class. Every class has a method apply which contains instruction for 
what's next to do. All these methods take the same arguments and (except 
in one case) return the same kind of tuple.

The interpret function uses 4 variables that are updated: expr, env, 
cont, val. Thus, the result of .apply(_,_,_,_) is the tuple for update.

class Contk(object):
     def __init__(self,*arg):
         raise NotImplementedError("For abstract class")

     def apply(self,expr,env,val):
         raise NotImplementedError("For abstract class")


class Endk(Contk):
     def __init__(self,val):
         self.val = val

     def apply(self,expr, env, val):
         return self.val

class Idk(Contk):
     def __init__(self):
         pass

     def apply(self, expr, env, val):
         return expr, env, Endk(val), val

There is a class for each continuation needed by the interpret.

The trouble is, I get this error:

[translation:ERROR]  AnnotatorError': annotation of 'union' degenerated 
to SomeObject()
[translation:ERROR] Simple call of incompatible family:
[translation:ERROR]       (KeyError getting at the binding!)
[translation:ERROR]
[translation:ERROR] In <FunctionGraph of 
(RPinterpretImperative:27)Idk.apply at 0x1b31be8>:
[translation:ERROR] Happened at file RPinterpretImperative.py line 28
[translation:ERROR]
[translation:ERROR] ==>         return expr, env, Endk(val), val
[translation:ERROR]
[translation:ERROR] Previous annotation:
[translation:ERROR]   (none)
[translation:ERROR]     .. v0 = simple_call((classobj Endk), val_0)
[translation:ERROR]     .. '(RPinterpretImperative:27)Idk.apply'
[translation:ERROR] Processing block:
[translation:ERROR]  block at 12 is a <class 
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (RPinterpretImperative:27)Idk.apply
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v0 = simple_call((classobj Endk), val_0)
[translation:ERROR]        v1 = newtuple(expr_0, env_0, v0, val_0)
[translation:ERROR]  --end--


So at first I thought it was due to tuple that would not accept newly 
created objects, but after a few tests, it appears that not. Then I 
tought the annotator couldn't realize by himself that returning an Endk 
object is the same as returning a Contk, so I annotated the whole file 
with assertion to force him to realize that my variable cont should be a 
Contk not any other kind of subclass, but it didn't work either.

So I don't know what to do now.

I've even tried to implement apply outside of the classes, or to make a 
function for each parameters returned, but nothing worked.

You can find the source here : 
https://github.com/zebign1/RPython-internship/tree/master/Interpreters/ifF1WAE
(treeClass.py and parser.py as tools, the bugged file is 
RPinterpretImperative.py)

Thanks again.

Leonard

From amauryfa at gmail.com  Mon Jun 25 17:10:20 2012
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 25 Jun 2012 17:10:20 +0200
Subject: [pypy-dev] Help (again)
In-Reply-To: <4FE87842.3070209@ens.fr>
References: <4FE87842.3070209@ens.fr>
Message-ID: <CAGmFidYsao6JCW-Jz45tnQfFijiXBxHoQA9dp+yfZenub2NK8w@mail.gmail.com>

2012/6/25 L?onard de Haro <leonard.de.haro at ens.fr>

> Hi everyone!
>
> First of all, thaks for your answers last time. Second of all, I need your
> help again.
>
> I'm coding an imperative version of my interpret and to do so, I coded
> continuations. I used a similat technique to the one used in the Prolog
> interpret you advised me to take a look at last time.
>
> I define a serie of class all inheriting from a simulated-abstract Contk
> class. Every class has a method apply which contains instruction for what's
> next to do. All these methods take the same arguments and (except in one
> case) return the same kind of tuple.
>
> The interpret function uses 4 variables that are updated: expr, env, cont,
> val. Thus, the result of .apply(_,_,_,_) is the tuple for update.
>
> class Contk(object):
>    def __init__(self,*arg):
>        raise NotImplementedError("For abstract class")
>
>    def apply(self,expr,env,val):
>        raise NotImplementedError("For abstract class")
>
>
> class Endk(Contk):
>    def __init__(self,val):
>        self.val = val
>
>    def apply(self,expr, env, val):
>        return self.val
>
> class Idk(Contk):
>    def __init__(self):
>        pass
>
>    def apply(self, expr, env, val):
>        return expr, env, Endk(val), val
>

Even if RPython needs no explicit declaration, its constraints are similar
to C++ or Java:
overridden methods must have a compatible signature, arguments and return
value.
Here, Idk.apply returns a 4-tuple, whereas Endk returned a single value.


> There is a class for each continuation needed by the interpret.
>
> The trouble is, I get this error:
>
> [translation:ERROR]  AnnotatorError': annotation of 'union' degenerated to
> SomeObject()
> [translation:ERROR] Simple call of incompatible family:
> [translation:ERROR]       (KeyError getting at the binding!)
> [translation:ERROR]
> [translation:ERROR] In <FunctionGraph of (RPinterpretImperative:27)Idk.**apply
> at 0x1b31be8>:
> [translation:ERROR] Happened at file RPinterpretImperative.py line 28
> [translation:ERROR]
> [translation:ERROR] ==>         return expr, env, Endk(val), val
> [translation:ERROR]
> [translation:ERROR] Previous annotation:
> [translation:ERROR]   (none)
> [translation:ERROR]     .. v0 = simple_call((classobj Endk), val_0)
> [translation:ERROR]     .. '(RPinterpretImperative:27)**Idk.apply'
> [translation:ERROR] Processing block:
> [translation:ERROR]  block at 12 is a <class 'pypy.objspace.flow.**
> flowcontext.SpamBlock'>
> [translation:ERROR]  in (RPinterpretImperative:27)Idk.**apply
> [translation:ERROR]  containing the following operations:
> [translation:ERROR]        v0 = simple_call((classobj Endk), val_0)
> [translation:ERROR]        v1 = newtuple(expr_0, env_0, v0, val_0)
> [translation:ERROR]  --end--
>
>
> So at first I thought it was due to tuple that would not accept newly
> created objects, but after a few tests, it appears that not. Then I tought
> the annotator couldn't realize by himself that returning an Endk object is
> the same as returning a Contk, so I annotated the whole file with assertion
> to force him to realize that my variable cont should be a Contk not any
> other kind of subclass, but it didn't work either.
>
> So I don't know what to do now.
>
> I've even tried to implement apply outside of the classes, or to make a
> function for each parameters returned, but nothing worked.
>
> You can find the source here : https://github.com/zebign1/**
> RPython-internship/tree/**master/Interpreters/ifF1WAE<https://github.com/zebign1/RPython-internship/tree/master/Interpreters/ifF1WAE>
> (treeClass.py and parser.py as tools, the bugged file is
> RPinterpretImperative.py)
>
> Thanks again.
>
> Leonard
> ______________________________**_________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>
>



-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120625/6e6cf3b9/attachment.html>

From Ronny.Pfannschmidt at gmx.de  Mon Jun 25 23:33:12 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Mon, 25 Jun 2012 23:33:12 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <4FE6D065.7030605@gmx.de>
References: <201206221305.30243.naylor.b.david@gmail.com>
	<4FE6D065.7030605@gmx.de>
Message-ID: <4FE8D918.4010504@gmx.de>

Hi David,

i created the kill-import_from_lib_pypy branch,
which should fix the issue - its still in testing,
please take a look if any issue comes up

-- Ronny

On 06/24/2012 10:31 AM, Ronny Pfannschmidt wrote:
> Hi David,
>
> your change to pylib introduces a worse bug thats not acceptable
>
> it really should error out in that case,
> since it would enforce a partial code reload of the old code or even a
> wrong code reload for subpackages
>
> -- Ronny
>
>
> On 06/22/2012 01:05 PM, David Naylor wrote:
>> Hi,
>>
>> I'm experimenting with using non-default library directories by
>> tweeking pypy.tool.lib_pypy however I came up with the following error
>> when translating with pypy:
>>
>> [translation:ERROR] Error:
>> [translation:ERROR] Traceback (most recent call last):
>> [translation:ERROR] File "translate.py", line 274, in main
>> [translation:ERROR] default_goal='compile')
>> [translation:ERROR] File
>> "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py",
>> line 810, in
>> from_targetspec
>> [translation:ERROR] spec = target(driver, args)
>> [translation:ERROR] File "targetpypystandalone.py", line 228, in target
>> [translation:ERROR] return self.get_entry_point(config)
>> [translation:ERROR] File "targetpypystandalone.py", line 236, in
>> get_entry_point
>> [translation:ERROR] rebuild =
>> import_from_lib_pypy('ctypes_config_cache/rebuild')
>> [translation:ERROR] File
>> "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/tool/lib_pypy.py",
>> line 12, in
>> import_from_lib_pypy
>> [translation:ERROR] return modname.pyimport()
>> [translation:ERROR] File
>> "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/py/_path/local.py",
>> line 531, in pyimport
>> [translation:ERROR] mod = __import__(modname, None, None, ['__doc__'])
>> [translation:ERROR] File
>> "/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/rebuild.py", line
>> 11, in<module>
>> [translation:ERROR] execfile(autopath_py, dict(__name__='autopath',
>> __file__=autopath_py))
>> [translation:ERROR] IOError: [Errno 2] No such file or directory:
>> '/usr/local/pypy-1.9/pypy/tool/autopath.py'
>>
>> This error does not happen when translating with python.
>>
>> Upon investigation I found that pypy has:
>> # pypy -c 'import sys; import pypy.tool.lib_pypy; print
>> "ctypes_config_cache" in sys.modules.keys()'
>> True
>>
>> but python has:
>> # python -c 'import sys; import pypy.tool.lib_pypy; print
>> "ctypes_config_cache" in sys.modules.keys()'
>> False
>>
>> also:
>> # pypy -c 'import sys; import pypy.tool.lib_pypy; print
>> sys.modules["ctypes_config_cache"]'
>> <module 'ctypes_config_cache' from
>> '/usr/local/pypy-1.9/lib_pypy/ctypes_config_cache/__init__.pyc'>
>> note the above always happens regardless of pypy.tool.lib_pypy being
>> modified or not.
>>
>> It appears to me that there are two bugs:
>> 1) pypy ends up importing ctypes_config_cache when it shouldn't???
>> 2) py/_path/local.py doesn't ensure import from correct location
>>
>> And attached is a patch for fixing (2), which works around (1).
>>
>> Regards
>>
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev


From celiaconno at caryon.com  Tue Jun 26 21:01:36 2012
From: celiaconno at caryon.com (Celia Connoley)
Date: Tue, 26 Jun 2012 20:01:36 +0100
Subject: [pypy-dev] Placement Site - Offer
Message-ID: <40827454.20120626200136@caryon.com>

pypy-dev at codespeak.net    :   


Your site has a good look and feel but you're not getting as much web traffic as you could be getting.  Can we show you how to change that?  It won't cost you anything for us to review your online business and the results could spike your web revenue.  Reply to us today and we will do a free assessment of your web presence.  Don't forget to include the best way we can contact you.
Sincerely,

Celia Connoley
  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120626/e2ed3d95/attachment.html>

From arigo at tunes.org  Wed Jun 27 09:59:59 2012
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 27 Jun 2012 09:59:59 +0200
Subject: [pypy-dev] [pypy/pypy] rbigint.py improvements (pull request
	#72)
In-Reply-To: <83984e591a7b9d585ea6b77b8bf276a1@bitbucket.org>
References: <83984e591a7b9d585ea6b77b8bf276a1@bitbucket.org>
Message-ID: <CAMSv6X1tz2-T+RD4m-t0T_RLkuQGW_zR5EqG4T=OC5yL8kLG4w@mail.gmail.com>

Hi Stian,

On Fri, Jun 22, 2012 at 3:59 AM, Stian Andreassen
<pullrequests-noreply at bitbucket.org> wrote:
> https://bitbucket.org/pypy/pypy/pull-request/72/rbigintpy-improvements
> Title: rbigint.py improvements

I suppose this needs discussion on pypy-dev.  It is great, but the
major issue is that it uses "__int128", which is an integer type that
is only sometimes available --- and more often not: it is present on
MSVC, but for GCC it requires 4.6 (and likely 64-bit machines, not
32-bit ones).  There is no way we can just merge it into trunk: it
would stop compiling on most of our buildbots, to start with.

So the discussion is about what to do from there.  Stian, I assume
you'd have to add the proper conditional compilation and support the
case of no int128, unless we can think of something better.


A bient?t,

Armin.

From naylor.b.david at gmail.com  Wed Jun 27 20:47:28 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Wed, 27 Jun 2012 20:47:28 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <4FE8D918.4010504@gmx.de>
References: <201206221305.30243.naylor.b.david@gmail.com>
	<4FE6D065.7030605@gmx.de> <4FE8D918.4010504@gmx.de>
Message-ID: <201206272047.34451.naylor.b.david@gmail.com>

On Monday, 25 June 2012 23:33:12 Ronny Pfannschmidt wrote:
> Hi David,
> 
> i created the kill-import_from_lib_pypy branch,
> which should fix the issue - its still in testing,
> please take a look if any issue comes up

Alas, the changes you made to pypy/tool/compat.py and pypy/translator/goal/targetpypystandalone.py didn't work in my setup...
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 274, in main
[translation:ERROR]     default_goal='compile')
[translation:ERROR]    File "/tmp/home/DragonSA/ports/pypy/work/pypy-pypy-341e1e3821ff/pypy/translator/driver.py", line 810, in 
from_targetspec
[translation:ERROR]     spec = target(driver, args)
[translation:ERROR]    File "targetpypystandalone.py", line 228, in target
[translation:ERROR]     return self.get_entry_point(config)
[translation:ERROR]    File "targetpypystandalone.py", line 235, in get_entry_point
[translation:ERROR]     from lib_pypy.ctypes_config_cache.rebuild import try_rebuild
[translation:ERROR]    File "/usr/local/lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py", line 11, in <module>
[translation:ERROR]     execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
[translation:ERROR]  IOError: [Errno 2] No such file or directory: '/usr/local/pypy/tool/autopath.py'

I have the following changes:
# cd pypy
# mkdir lib
# mv lib-python/2.7 lib/pypy1.9
# mv lib_pypy lib/pypy1.9/lib_pypy
# patch << _EOF
--- lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py~        2012-06-22 11:42:55.000000000 +0200
+++ lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py 2012-06-22 11:43:12.000000000 +0200
@@ -6,7 +6,7 @@
 # get the correct path
 import os.path
 this_dir = os.path.dirname(__file__)
-autopath_py = os.path.join(this_dir, '../../pypy/tool/autopath.py')
+autopath_py = os.path.join(this_dir, '../../../../pypy/tool/autopath.py')
 autopath_py = os.path.abspath(autopath_py)
 execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
 
--- pypy/module/sys/state.py.orig       2012-06-07 14:24:48.000000000 +0200
+++ pypy/module/sys/state.py    2012-06-22 14:45:23.000000000 +0200
@@ -36,14 +36,12 @@
 platform = sys.platform
 
 def getinitialpath(state, prefix):
-    from pypy.module.sys.version import CPYTHON_VERSION
-    dirname = '%d.%d' % (CPYTHON_VERSION[0],
-                         CPYTHON_VERSION[1])
-    lib_python = os.path.join(prefix, 'lib-python')
-    python_std_lib = os.path.join(lib_python, dirname)
+    from pypy.module.sys.version import PYPY_VERSION
+    libpath = os.path.join(prefix, 'lib')
+    python_std_lib = os.path.join(libpath, 'pypy%d.%d' % PYPY_VERSION[:2])
     checkdir(python_std_lib)
     
-    lib_pypy = os.path.join(prefix, 'lib_pypy')
+    lib_pypy = os.path.join(python_std_lib, 'lib_pypy')
     checkdir(lib_pypy)
 
     importlist = []
--- pypy/tool/lib_pypy.py.orig  2012-06-07 14:24:48.000000000 +0200
+++ pypy/tool/lib_pypy.py       2012-06-22 14:46:42.000000000 +0200
@@ -1,12 +1,11 @@
 import py
 import pypy
 import pypy.module
-from pypy.module.sys.version import CPYTHON_VERSION
+from pypy.module.sys.version import PYPY_VERSION
 
 LIB_ROOT = py.path.local(pypy.__path__[0]).dirpath()
-LIB_PYPY =  LIB_ROOT.join('lib_pypy')
-LIB_PYTHON = LIB_ROOT.join('lib-python', '%d.%d' % CPYTHON_VERSION[:2])
-
+LIB_PYTHON =  LIB_ROOT.join('lib', 'pypy%d.%d' % PYPY_VERSION[:2])
+LIB_PYPY = LIB_PYTHON.join('lib_pypy')
 
 def import_from_lib_pypy(modname):
     modname = LIB_PYPY.join(modname+'.py')
_EOF

and with the following patch the translate works successfully for me:
# patch << _EOF
--- py/_path/local.py~  2012-06-22 12:20:36.000000000 +0200
+++ py/_path/local.py   2012-06-22 12:21:45.000000000 +0200
@@ -516,6 +516,9 @@
             pkgpath = self.pypkgpath()
             if pkgpath is not None:
                 if ensuresyspath:
+                    import sys
+                    if pkgpath.basename in sys.modules:
+                        del sys.modules[pkgpath.basename]
                     self._prependsyspath(pkgpath.dirpath())
                 pkg = __import__(pkgpath.basename, None, None, [])
                 names = self.new(ext='').relto(pkgpath.dirpath())
_EOF

Sorry, I should have mentioned this is my earlier post...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120627/7781fb08/attachment.pgp>

From naylor.b.david at gmail.com  Wed Jun 27 21:19:00 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Wed, 27 Jun 2012 21:19:00 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <201206272047.34451.naylor.b.david@gmail.com>
References: <201206221305.30243.naylor.b.david@gmail.com>
	<4FE8D918.4010504@gmx.de>
	<201206272047.34451.naylor.b.david@gmail.com>
Message-ID: <201206272119.02946.naylor.b.david@gmail.com>

On Wednesday, 27 June 2012 20:47:28 David Naylor wrote:
> On Monday, 25 June 2012 23:33:12 Ronny Pfannschmidt wrote:
> > Hi David,
> > 
> > i created the kill-import_from_lib_pypy branch,
> > which should fix the issue - its still in testing,
> > please take a look if any issue comes up

With the changes below your patch works (also, unpatched the translation 
works, whereas before it did not).  

> Alas, the changes you made to pypy/tool/compat.py and
> pypy/translator/goal/targetpypystandalone.py didn't work in my setup...

I believe that, due to my relocation of lib_pypy, I didn't catch all the cases 
(specifically autopath.py), and due to my error these errors have been 
occuring... Sorry!

> I have the following changes:

And these additions to get a "properly" working environment

> # cd pypy
> # mkdir lib
> # mv lib-python/2.7 lib/pypy1.9
> # mv lib_pypy lib/pypy1.9/lib_pypy

# ln lib/pypy1.9/lib_pypy lib_pypy

> # patch << _EOF
> --- lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py~        2012-06-22
> 11:42:55.000000000 +0200 +++
> lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py 2012-06-22
> 11:43:12.000000000 +0200 @@ -6,7 +6,7 @@
>  # get the correct path
>  import os.path
>  this_dir = os.path.dirname(__file__)
> -autopath_py = os.path.join(this_dir, '../../pypy/tool/autopath.py')
> +autopath_py = os.path.join(this_dir, '../../../../pypy/tool/autopath.py')
>  autopath_py = os.path.abspath(autopath_py)
>  execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
> 

Using your patches the above change is not needed.  

> --- pypy/module/sys/state.py.orig       2012-06-07 14:24:48.000000000 +0200
> +++ pypy/module/sys/state.py    2012-06-22 14:45:23.000000000 +0200
> @@ -36,14 +36,12 @@
>  platform = sys.platform
> 
>  def getinitialpath(state, prefix):
> -    from pypy.module.sys.version import CPYTHON_VERSION
> -    dirname = '%d.%d' % (CPYTHON_VERSION[0],
> -                         CPYTHON_VERSION[1])
> -    lib_python = os.path.join(prefix, 'lib-python')
> -    python_std_lib = os.path.join(lib_python, dirname)
> +    from pypy.module.sys.version import PYPY_VERSION
> +    libpath = os.path.join(prefix, 'lib')
> +    python_std_lib = os.path.join(libpath, 'pypy%d.%d' % PYPY_VERSION[:2])
>      checkdir(python_std_lib)
> 
> -    lib_pypy = os.path.join(prefix, 'lib_pypy')
> +    lib_pypy = os.path.join(python_std_lib, 'lib_pypy')
>      checkdir(lib_pypy)
> 
>      importlist = []
> --- pypy/tool/lib_pypy.py.orig  2012-06-07 14:24:48.000000000 +0200
> +++ pypy/tool/lib_pypy.py       2012-06-22 14:46:42.000000000 +0200
> @@ -1,12 +1,11 @@
>  import py
>  import pypy
>  import pypy.module
> -from pypy.module.sys.version import CPYTHON_VERSION
> +from pypy.module.sys.version import PYPY_VERSION
> 
>  LIB_ROOT = py.path.local(pypy.__path__[0]).dirpath()
> -LIB_PYPY =  LIB_ROOT.join('lib_pypy')
> -LIB_PYTHON = LIB_ROOT.join('lib-python', '%d.%d' % CPYTHON_VERSION[:2])
> -
> +LIB_PYTHON =  LIB_ROOT.join('lib', 'pypy%d.%d' % PYPY_VERSION[:2])
> +LIB_PYPY = LIB_PYTHON.join('lib_pypy')
> 
>  def import_from_lib_pypy(modname):
>      modname = LIB_PYPY.join(modname+'.py')
> _EOF

And with just the above translation works successfully for me.

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120627/6f1d9ed7/attachment.pgp>

From paulo.koch at gmail.com  Thu Jun 28 18:58:17 2012
From: paulo.koch at gmail.com (=?UTF-8?Q?Paulo_K=C3=B6ch?=)
Date: Thu, 28 Jun 2012 17:58:17 +0100
Subject: [pypy-dev] Hello! and Video Tutorials
Message-ID: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>

Hello!

My name is Paulo K?ch. I'm from Portugal and looking to contribute to PyPy.
Right now, I'm still just digesting the available docs. I'll bet back to
you as soon as I feel informed enough to start hacking.

In my quest for documentation, I stumbled on the videos and their torrents.
However, the tracker at http://codespeak.net:80/ always replies with
something that my BT client doesn't understand. Going there with a browser,
the site seems to be somewhat defunct, containing just links to other sites.

So:
*  Where can I find these videos?
*  Why aren't they on youtube or something?

Cheers!
Paulo K?ch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120628/7474baba/attachment.html>

From anto.cuni at gmail.com  Fri Jun 29 01:08:28 2012
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 29 Jun 2012 01:08:28 +0200
Subject: [pypy-dev] Developer selection for Py3k
Message-ID: <4FECE3EC.7030000@gmail.com>

Hi all,

as you probably know, the Py3k [1] proposal is getting funded thanks to our
generous donors.

During the first round, three of use were selected: me, Alex Gaynor and
Benjamin Peterson.  However, due to unforeseen unavailability of Alex and
Benjamin, we are now looking for one more developer to help with the py3k work.

If you are interested in getting paid to work on the Py3k proposal, please
apply by replying to this email.

To be applicable you need to be an experienced PyPy developer, preferably with
a previous experience on the Python Interpreter part of PyPy.

ciao,
Anto

From arigo at tunes.org  Fri Jun 29 16:09:06 2012
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 29 Jun 2012 16:09:06 +0200
Subject: [pypy-dev] Hello! and Video Tutorials
In-Reply-To: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
References: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
Message-ID: <CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>

Hi Paulo,

On Thu, Jun 28, 2012 at 6:58 PM, Paulo K?ch <paulo.koch at gmail.com> wrote:
> * ?Where can I find these videos?
> * ?Why aren't they on youtube or something?

It turns out that the videos are available directly from
http://codespeak.net/download/video-pypy/ .  To Holger: I don't know
if this is an oversight or not.  Should we instead just move them to
youtube?  (Mostly for historical value)


A bient?t,

Armin.

From paulo.koch at gmail.com  Fri Jun 29 17:52:39 2012
From: paulo.koch at gmail.com (=?UTF-8?Q?Paulo_K=C3=B6ch?=)
Date: Fri, 29 Jun 2012 16:52:39 +0100
Subject: [pypy-dev] Hello! and Video Tutorials
In-Reply-To: <CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
References: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
	<CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
Message-ID: <CAM_z3tGf4v-TMd-xQTnnWcUQbtkOqhXGPaeffVwHhQUu9jSg7w@mail.gmail.com>

On Fri, Jun 29, 2012 at 3:09 PM, Armin Rigo <arigo at tunes.org> wrote:
> It turns out that the videos are available directly from
> http://codespeak.net/download/video-pypy/ .

Thank you! :)

> To Holger: I don't know
> if this is an oversight or not. ?Should we instead just move them to
> youtube? ?(Mostly for historical value)

Holger: I can volunteer to upload them, if you like.

From holger at merlinux.eu  Fri Jun 29 19:39:18 2012
From: holger at merlinux.eu (holger krekel)
Date: Fri, 29 Jun 2012 17:39:18 +0000
Subject: [pypy-dev] Hello! and Video Tutorials
In-Reply-To: <CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
References: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
	<CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
Message-ID: <20120629173918.GX11942@merlinux.eu>

On Fri, Jun 29, 2012 at 16:09 +0200, Armin Rigo wrote:
> On Thu, Jun 28, 2012 at 6:58 PM, Paulo K?ch <paulo.koch at gmail.com> wrote:
> > * ?Where can I find these videos?
> > * ?Why aren't they on youtube or something?
> 
> It turns out that the videos are available directly from
> http://codespeak.net/download/video-pypy/ .  To Holger: I don't know
> if this is an oversight or not.  Should we instead just move them to
> youtube?  (Mostly for historical value)

If somebody can move them to youtube, this sounds like a good idea!

holger

> 
> A bient?t,
> 
> Armin.
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
> 

From paulo.koch at gmail.com  Fri Jun 29 19:51:22 2012
From: paulo.koch at gmail.com (=?UTF-8?Q?Paulo_K=C3=B6ch?=)
Date: Fri, 29 Jun 2012 18:51:22 +0100
Subject: [pypy-dev] Hello! and Video Tutorials
In-Reply-To: <20120629173918.GX11942@merlinux.eu>
References: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
	<CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
	<20120629173918.GX11942@merlinux.eu>
Message-ID: <CAM_z3tF56ENatjJNe+1NYA+p+oTMPXb+oDsdFJivhEdzB-FFcQ@mail.gmail.com>

On Fri, Jun 29, 2012 at 6:39 PM, holger krekel <holger at merlinux.eu> wrote:
> If somebody can move them to youtube, this sounds like a good idea!

Should I create a user for the PyPy project or is there one already?
How do we go about keeping the password?

From pjenvey at underboss.org  Fri Jun 29 21:39:05 2012
From: pjenvey at underboss.org (Philip Jenvey)
Date: Fri, 29 Jun 2012 12:39:05 -0700
Subject: [pypy-dev] Developer selection for Py3k
In-Reply-To: <4FECE3EC.7030000@gmail.com>
References: <4FECE3EC.7030000@gmail.com>
Message-ID: <AC19A426-F94E-4D68-97C3-E397CF2D9F5F@underboss.org>


On Jun 28, 2012, at 4:08 PM, Antonio Cuni wrote:

> Hi all,
> 
> as you probably know, the Py3k [1] proposal is getting funded thanks to our
> generous donors.
> 
> During the first round, three of use were selected: me, Alex Gaynor and
> Benjamin Peterson.  However, due to unforeseen unavailability of Alex and
> Benjamin, we are now looking for one more developer to help with the py3k work.
> 
> If you are interested in getting paid to work on the Py3k proposal, please
> apply by replying to this email.
> 
> To be applicable you need to be an experienced PyPy developer, preferably with
> a previous experience on the Python Interpreter part of PyPy.

If it was a few months ago I'd be up for this, unfortunately I'm not available now. I'll let you know if that changes.

--
Philip Jenvey


From Ronny.Pfannschmidt at gmx.de  Fri Jun 29 21:46:58 2012
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Fri, 29 Jun 2012 21:46:58 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <201206272119.02946.naylor.b.david@gmail.com>
References: <201206221305.30243.naylor.b.david@gmail.com>
	<4FE8D918.4010504@gmx.de>
	<201206272047.34451.naylor.b.david@gmail.com>
	<201206272119.02946.naylor.b.david@gmail.com>
Message-ID: <4FEE0632.9080402@gmx.de>

Hi David,

i proceeded with refining my changes,
moving all code related to writing these dumps out of lib_pypy

i'd like to know why you are relocating lib_pypy/lib-python

-- Ronny

On 06/27/2012 09:19 PM, David Naylor wrote:
> On Wednesday, 27 June 2012 20:47:28 David Naylor wrote:
>> On Monday, 25 June 2012 23:33:12 Ronny Pfannschmidt wrote:
>>> Hi David,
>>>
>>> i created the kill-import_from_lib_pypy branch,
>>> which should fix the issue - its still in testing,
>>> please take a look if any issue comes up
>
> With the changes below your patch works (also, unpatched the translation
> works, whereas before it did not).
>
>> Alas, the changes you made to pypy/tool/compat.py and
>> pypy/translator/goal/targetpypystandalone.py didn't work in my setup...
>
> I believe that, due to my relocation of lib_pypy, I didn't catch all the cases
> (specifically autopath.py), and due to my error these errors have been
> occuring... Sorry!
>
>> I have the following changes:
>
> And these additions to get a "properly" working environment
>
>> # cd pypy
>> # mkdir lib
>> # mv lib-python/2.7 lib/pypy1.9
>> # mv lib_pypy lib/pypy1.9/lib_pypy
>
> # ln lib/pypy1.9/lib_pypy lib_pypy
>
>> # patch<<  _EOF
>> --- lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py~        2012-06-22
>> 11:42:55.000000000 +0200 +++
>> lib/pypy1.9/lib_pypy/ctypes_config_cache/rebuild.py 2012-06-22
>> 11:43:12.000000000 +0200 @@ -6,7 +6,7 @@
>>   # get the correct path
>>   import os.path
>>   this_dir = os.path.dirname(__file__)
>> -autopath_py = os.path.join(this_dir, '../../pypy/tool/autopath.py')
>> +autopath_py = os.path.join(this_dir, '../../../../pypy/tool/autopath.py')
>>   autopath_py = os.path.abspath(autopath_py)
>>   execfile(autopath_py, dict(__name__='autopath', __file__=autopath_py))
>>
>
> Using your patches the above change is not needed.
>
>> --- pypy/module/sys/state.py.orig       2012-06-07 14:24:48.000000000 +0200
>> +++ pypy/module/sys/state.py    2012-06-22 14:45:23.000000000 +0200
>> @@ -36,14 +36,12 @@
>>   platform = sys.platform
>>
>>   def getinitialpath(state, prefix):
>> -    from pypy.module.sys.version import CPYTHON_VERSION
>> -    dirname = '%d.%d' % (CPYTHON_VERSION[0],
>> -                         CPYTHON_VERSION[1])
>> -    lib_python = os.path.join(prefix, 'lib-python')
>> -    python_std_lib = os.path.join(lib_python, dirname)
>> +    from pypy.module.sys.version import PYPY_VERSION
>> +    libpath = os.path.join(prefix, 'lib')
>> +    python_std_lib = os.path.join(libpath, 'pypy%d.%d' % PYPY_VERSION[:2])
>>       checkdir(python_std_lib)
>>
>> -    lib_pypy = os.path.join(prefix, 'lib_pypy')
>> +    lib_pypy = os.path.join(python_std_lib, 'lib_pypy')
>>       checkdir(lib_pypy)
>>
>>       importlist = []
>> --- pypy/tool/lib_pypy.py.orig  2012-06-07 14:24:48.000000000 +0200
>> +++ pypy/tool/lib_pypy.py       2012-06-22 14:46:42.000000000 +0200
>> @@ -1,12 +1,11 @@
>>   import py
>>   import pypy
>>   import pypy.module
>> -from pypy.module.sys.version import CPYTHON_VERSION
>> +from pypy.module.sys.version import PYPY_VERSION
>>
>>   LIB_ROOT = py.path.local(pypy.__path__[0]).dirpath()
>> -LIB_PYPY =  LIB_ROOT.join('lib_pypy')
>> -LIB_PYTHON = LIB_ROOT.join('lib-python', '%d.%d' % CPYTHON_VERSION[:2])
>> -
>> +LIB_PYTHON =  LIB_ROOT.join('lib', 'pypy%d.%d' % PYPY_VERSION[:2])
>> +LIB_PYPY = LIB_PYTHON.join('lib_pypy')
>>
>>   def import_from_lib_pypy(modname):
>>       modname = LIB_PYPY.join(modname+'.py')
>> _EOF
>
> And with just the above translation works successfully for me.
>
> Regards


From piotr.skamruk at gmail.com  Fri Jun 29 21:51:59 2012
From: piotr.skamruk at gmail.com (Piotr Skamruk)
Date: Fri, 29 Jun 2012 21:51:59 +0200
Subject: [pypy-dev] Developer selection for Py3k
In-Reply-To: <AC19A426-F94E-4D68-97C3-E397CF2D9F5F@underboss.org>
References: <4FECE3EC.7030000@gmail.com>
	<AC19A426-F94E-4D68-97C3-E397CF2D9F5F@underboss.org>
Message-ID: <CAO_YSZMAgnKJwrjzBKmOojqxqmoMAO9oNcuJfB1eoSWLXbMMAA@mail.gmail.com>

heh, now i'm using both python 3.x (under pyramid applications) and
pypy (under twisted application) and i would be happy to help with
connecting this two aspects...
... but ... i'm not so competent to do that with money for consideration.

... second but - if there would be some dissected, small task, with
description about what have to be done - i will be happy to help in
squashing them from bugtracker.

so would some managing person slice milestone points to such (possibly
small, easy) tasks?

2012/6/29 Philip Jenvey <pjenvey at underboss.org>:
>
> On Jun 28, 2012, at 4:08 PM, Antonio Cuni wrote:
>
>> Hi all,
>>
>> as you probably know, the Py3k [1] proposal is getting funded thanks to our
>> generous donors.
>>
>> During the first round, three of use were selected: me, Alex Gaynor and
>> Benjamin Peterson. ?However, due to unforeseen unavailability of Alex and
>> Benjamin, we are now looking for one more developer to help with the py3k work.
>>
>> If you are interested in getting paid to work on the Py3k proposal, please
>> apply by replying to this email.
>>
>> To be applicable you need to be an experienced PyPy developer, preferably with
>> a previous experience on the Python Interpreter part of PyPy.
>
> If it was a few months ago I'd be up for this, unfortunately I'm not available now. I'll let you know if that changes.
>
> --
> Philip Jenvey
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev

From tismer at stackless.com  Sat Jun 30 01:33:50 2012
From: tismer at stackless.com (Christian Tismer)
Date: Sat, 30 Jun 2012 01:33:50 +0200
Subject: [pypy-dev] Developer selection for Py3k
In-Reply-To: <4FECE3EC.7030000@gmail.com>
References: <4FECE3EC.7030000@gmail.com>
Message-ID: <4FEE3B5E.4060206@stackless.com>

On 29.06.12 01:08, Antonio Cuni wrote:
> Hi all,
>
> as you probably know, the Py3k [1] proposal is getting funded thanks to our
> generous donors.
>
> During the first round, three of use were selected: me, Alex Gaynor and
> Benjamin Peterson.  However, due to unforeseen unavailability of Alex and
> Benjamin, we are now looking for one more developer to help with the py3k work.
>
> If you are interested in getting paid to work on the Py3k proposal, please
> apply by replying to this email.
>
> To be applicable you need to be an experienced PyPy developer, preferably with
> a previous experience on the Python Interpreter part of PyPy.
>

Hi Anto,

Starting from August, I have some time that I could spend on this.

cheers -- Chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
work +49 173 24 18 776  mobile +49 173 24 18 776  fax n.a.
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/


From ski at dag-system.com  Sat Jun 30 05:36:46 2012
From: ski at dag-system.com (Mud - Obstacles - Beer & Music)
Date: Sat, 30 Jun 2012 05:36:46 +0200 (CEST)
Subject: [pypy-dev] 5k mud runner &  2k family mud runner
Message-ID: <87382885.01341027406985@dag-system.com>

                              v\:* {behavior:url(#default#VML);}o\:* {behavior:url(#default#VML);}w\:* {behavior:url(#default#VML);}.shape {behavior:url(#default#VML);}   NormalEmail.dotm  1  0  2012-06-07T19:39:00Z  1  1038  5714     47  13  6739  12.00    0      21    false  false  false    FR  X-NONE  X-NONE                              MicrosoftInternetExplorer4                                                                                                                                                                                                                                                                                                                          /* Style Definitions */ table.MsoNormalTable	{mso-style-name:"Tableau Normal";	mso-tstyle-rowband-size:0;	mso-tstyle-colband-size:0;	mso-style-noshow:yes;	mso-style-priority:99;	mso-style-qformat:yes;	mso-style-parent:"";	mso-padding-alt:0cm 5.4pt 0cm 5.4pt;	mso-para-margin:0cm;	mso-para-margin-bottom:.0001pt;	mso-pagination:widow-orphan;	font-size:10.0pt;	font-family:"Times New Roman","serif";}      &#160;           

          More information: click here.          

          Testing your endurance is only the beginning. Having fun with friends, family and mud is pretty awesome and the after PARTY promises to be ... well a party. Music, rockin out on a Saturday afternoon, Cold beer and other beverages, food for body and soul (that's the beer). Yup and your not gonna find grandmas music either. This is your party...                        OK...We know that Maple Ski Ridge's vertical will not leave you breathless. Elevation, yup we have some... But for what we lack in straight up- we will more than make up for in sheer craziness in "Features"; Mud..water.. ledges..tires(obligatory).. hills..slop and all other types of things for you to go over -thru -around -and under.            &#160;            We don't think you will be disappointed.                        What should you bring???? Unless Great Grams parties, leave her at home. White T shirts that you no longer love are OK.&#160; Better yet, your younger sisters/brothers clothes are a great idea... And a towel to dry off. Yup and if you need clothes to party in after...                        More info will be posted here as we go. Such as music line up, how many registrations and pictures of the event in the making.&#160; Stay tuned....and sign up now.                       

              
        &#160;    

          &#160;WHEN:             &#160;&#160; Saturday July 28th 2012. Need we say rain or shine. MUD!          

          9:30 Streaching warm up          

          10:00 Briefing          

          10:15 5k mud start          

          11:00 2k family mud start          

          11:30 Streaching relaxation          

          12:00 Prices           

          12:30 Costumes awards          

          12:30 BBQ + Cold beer + Music (live band)          

          &#160;          

          &#160;          

          &#160;          

        WHERE:&#160;&#160;&#160;&#160; Maple Ski Ridge&#160;&#160; 2725 Mariaville Road&#160;&#160; Schenectady NY 12306      
      &#160;HOW?&#160;&#160;&#160;             Registration as a participant: Click here.&#160;&#160;&#160;           

        &#160;HOW MUCH?&#160;           

          5K Mud Run -&#160; $45.00 pp. First 100 get free Run the Ridge t-shirt, beverage and dog.             2K Family Mud Run $20.00 per adult and $10.00 per munchkin(10 and under).                       

              
      &#160;          

        &#160;      
        &#160;    

          Sponsor opportunities:          

          For your company, for your products, for your team, advertise your brand (TV, Logos, Booth, Tent...). Or just because Run the Ridge is currently seeking sponsors to build the best and most interesting event.           

          Click here, we will send you our "Run the Ridge sponsorship brochure".          

          Register now as a sponsor: Click here.          

          &#160;          

              
        &#160;&#160;    

    Powered by                 

        

    &#160;    

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120630/c6b390a2/attachment-0001.html>

From naylor.b.david at gmail.com  Sat Jun 30 17:59:46 2012
From: naylor.b.david at gmail.com (David Naylor)
Date: Sat, 30 Jun 2012 17:59:46 +0200
Subject: [pypy-dev] Translation with pypy picking up wrong libraries
In-Reply-To: <4FEE0632.9080402@gmx.de>
References: <201206221305.30243.naylor.b.david@gmail.com>
	<201206272119.02946.naylor.b.david@gmail.com>
	<4FEE0632.9080402@gmx.de>
Message-ID: <201206301759.50995.naylor.b.david@gmail.com>

On Friday, 29 June 2012 21:46:58 Ronny Pfannschmidt wrote:
> Hi David,

Hi

> i proceeded with refining my changes,
> moving all code related to writing these dumps out of lib_pypy

Thanks

> i'd like to know why you are relocating lib_pypy/lib-python

I am slowly integrating pypy into FreeBSD Port's Collection, with the eventual 
goal of installing third party python based ports (i.e. setup.py and such).  
FreeBSD follows a standard (I think) posix directory layout.  As such I wanted 
the pypy binary to live in $PREFIX/bin and the libraries in $PREFIX/lib.  

The convention for cPython is to have the libraries as $PREFIX/lib/python2.7, 
so for pypy I wanted to have $PREFIX/lib/pypy1.9 (although I did consider 
$PREFIX/lib/pypython2.7), which leaves the lib_pypy libraries.  These cannot 
be renamed to anything such as pypy1.9 so I stuck them within the standard 
libraries directory.  A condition for this is the libraries cannot conflict 
with cPython.  

Previously I had an explicit directory $PREFIX/pypy-1.9 where I had all the 
files (including the binary) with sym-links for the binaries to $PREFIX/bin.  
This, however, I considered less clean.  

Regards
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120630/8d56a1f7/attachment.pgp>

From fijall at gmail.com  Sat Jun 30 19:10:27 2012
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 30 Jun 2012 19:10:27 +0200
Subject: [pypy-dev] Hello! and Video Tutorials
In-Reply-To: <CAM_z3tF56ENatjJNe+1NYA+p+oTMPXb+oDsdFJivhEdzB-FFcQ@mail.gmail.com>
References: <CAM_z3tEowiBtxnGjDBgisHZ5v4-4-bEKiqZ6H1PMBtiYPjh+HQ@mail.gmail.com>
	<CAMSv6X35mpdqG4wSRMJnz4T08ymLTxRbv+Hs9iSCPO8hAMRqwQ@mail.gmail.com>
	<20120629173918.GX11942@merlinux.eu>
	<CAM_z3tF56ENatjJNe+1NYA+p+oTMPXb+oDsdFJivhEdzB-FFcQ@mail.gmail.com>
Message-ID: <CAK5idxRFEGOvHNq0fsNpo46QuEyg3Zz-jr08XbKb8JJAWQF-fg@mail.gmail.com>

On Fri, Jun 29, 2012 at 7:51 PM, Paulo K?ch <paulo.koch at gmail.com> wrote:

> On Fri, Jun 29, 2012 at 6:39 PM, holger krekel <holger at merlinux.eu> wrote:
> > If somebody can move them to youtube, this sounds like a good idea!
>
> Should I create a user for the PyPy project or is there one already?
> How do we go about keeping the password?


Just make up a random password, we'll get it from you later. There is no
pypy user as far as I know

Cheers,
fijal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120630/816bc462/attachment.html>