From cfbolz at gmx.de Tue Oct 1 17:48:51 2013 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 01 Oct 2013 17:48:51 +0200 Subject: [pypy-dev] Possible Numpy benchmark? Message-ID: Hi all, Maybe we can use this thing as a Numpy benchmark? http://radimrehurek.com/2013/09/deep-learning-with-word2vec-and-gensim/ Cheers, Carl Friedrich -------------- next part -------------- An HTML attachment was scrubbed... URL: From rjnienaber at gmail.com Tue Oct 1 21:59:07 2013 From: rjnienaber at gmail.com (Richard Nienaber) Date: Tue, 1 Oct 2013 20:59:07 +0100 Subject: [pypy-dev] Possible Numpy benchmark? In-Reply-To: References: Message-ID: Speaking of benchmarks, is there a chance this could be reviewed: https://bitbucket.org/pypy/benchmarks/pull-request/5/added-bench-mark-groups-and-some-more/diff Richard On Tue, Oct 1, 2013 at 4:48 PM, Carl Friedrich Bolz wrote: > Hi all, > > Maybe we can use this thing as a Numpy benchmark? > > http://radimrehurek.com/2013/09/deep-learning-with-word2vec-and-gensim/ > > Cheers, > > Carl Friedrich > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrish at pianocktail.org Tue Oct 1 23:04:33 2013 From: chrish at pianocktail.org (Christian Hudon) Date: Tue, 01 Oct 2013 17:04:33 -0400 Subject: [pypy-dev] stdlib-2.7.5 branch Message-ID: <4e6e3b9fd64a051b4cefc9b6c2746f11@chrish.pianocktail.org> Hello, I created a pull request for a fix for a failing json test in the stdlib-2.7.5 branch. This is my first pull request outside of a PyCon sprint. When at a sprint, I either put the name of the PyPy developer that had coached me in the reviewer field of the pull request, or asked them for a suggestion for a reviewer. So for that first patch outside of a PyPy sprint, I'm not quite sure what the right procedure is... What should I do in the future to get code reviews for my pull requests. Should I "just wait and someone will get to it (eventually)", or would it be better to ping people a bit more explicitly? Here is the pull request: https://bitbucket.org/pypy/pypy/pull-request/192/fix-for-failing-json-test/diff Also, any reason why buildbot isn't running automatically on the stdlib-2.7.5 branch? The last run was on August 31st. This is the branch which, once all tests pass, will be merged back into the trunk and then the next release of PyPy will be announced as supporting CPython 2.7.5, right? (Just making sure...) Assuming that's true, I'll work slowly on making all the tests pass, assuming nobody beats me to it. Thanks, Christian From pjenvey at underboss.org Wed Oct 2 00:51:03 2013 From: pjenvey at underboss.org (Philip Jenvey) Date: Tue, 1 Oct 2013 15:51:03 -0700 Subject: [pypy-dev] stdlib-2.7.5 branch In-Reply-To: <4e6e3b9fd64a051b4cefc9b6c2746f11@chrish.pianocktail.org> References: <4e6e3b9fd64a051b4cefc9b6c2746f11@chrish.pianocktail.org> Message-ID: On Tue, Oct 1, 2013 at 2:04 PM, Christian Hudon wrote: > Hello, > > I created a pull request for a fix for a failing json test in the > stdlib-2.7.5 branch. This is my first pull request outside of a PyCon > sprint. When at a sprint, I either put the name of the PyPy developer that > had coached me in the reviewer field of the pull request, or asked them for > a suggestion for a reviewer. So for that first patch outside of a PyPy > sprint, I'm not quite sure what the right procedure is... What should I do > in the future to get code reviews for my pull requests. Should I "just wait > and someone will get to it (eventually)", or would it be better to ping > people a bit more explicitly? > I'd ping someone on IRC or the ML as our response time on pull requests could probably be better =] > > Here is the pull request: https://bitbucket.org/pypy/** > pypy/pull-request/192/fix-for-**failing-json-test/diff > > Also, any reason why buildbot isn't running automatically on the > stdlib-2.7.5 branch? The last run was on August 31st. This is the branch > which, once all tests pass, will be merged back into the trunk and then the > next release of PyPy will be announced as supporting CPython 2.7.5, right? > (Just making sure...) Assuming that's true, I'll work slowly on making all > the tests pass, assuming nobody beats me to it. > No good reason, it just requires someone to change the pypy buildbot codebase and ensure someone with access propagates the changes. I don't know if anyone's set up automatic builds for stdlib release branches like this in the past. FYI you (or anyone) can manually trigger builds on your own. -- Philip Jenvey -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrewsmedina at gmail.com Wed Oct 2 01:14:28 2013 From: andrewsmedina at gmail.com (Andrews Medina) Date: Tue, 1 Oct 2013 20:14:28 -0300 Subject: [pypy-dev] stdlib-2.7.5 branch In-Reply-To: References: <4e6e3b9fd64a051b4cefc9b6c2746f11@chrish.pianocktail.org> Message-ID: I think that the build is running automatically. The last commit on this branch was in 2013-08-31 On Tue, Oct 1, 2013 at 7:51 PM, Philip Jenvey wrote: > On Tue, Oct 1, 2013 at 2:04 PM, Christian Hudon > wrote: >> >> Hello, >> >> I created a pull request for a fix for a failing json test in the >> stdlib-2.7.5 branch. This is my first pull request outside of a PyCon >> sprint. When at a sprint, I either put the name of the PyPy developer that >> had coached me in the reviewer field of the pull request, or asked them for >> a suggestion for a reviewer. So for that first patch outside of a PyPy >> sprint, I'm not quite sure what the right procedure is... What should I do >> in the future to get code reviews for my pull requests. Should I "just wait >> and someone will get to it (eventually)", or would it be better to ping >> people a bit more explicitly? > > > I'd ping someone on IRC or the ML as our response time on pull requests > could probably be better =] > >> >> >> Here is the pull request: >> https://bitbucket.org/pypy/pypy/pull-request/192/fix-for-failing-json-test/diff >> >> Also, any reason why buildbot isn't running automatically on the >> stdlib-2.7.5 branch? The last run was on August 31st. This is the branch >> which, once all tests pass, will be merged back into the trunk and then the >> next release of PyPy will be announced as supporting CPython 2.7.5, right? >> (Just making sure...) Assuming that's true, I'll work slowly on making all >> the tests pass, assuming nobody beats me to it. > > > No good reason, it just requires someone to change the pypy buildbot > codebase and ensure someone with access propagates the changes. I don't know > if anyone's set up automatic builds for stdlib release branches like this in > the past. > > FYI you (or anyone) can manually trigger builds on your own. > > -- > Philip Jenvey > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- Andrews Medina www.andrewsmedina.com From rami.chowdhury at gmail.com Sun Oct 6 19:33:58 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Sun, 6 Oct 2013 18:33:58 +0100 Subject: [pypy-dev] stdlib-2.7.5 branch In-Reply-To: References: <4e6e3b9fd64a051b4cefc9b6c2746f11@chrish.pianocktail.org> Message-ID: On Tuesday, October 1, 2013 at 11:51 PM, Philip Jenvey wrote: > On Tue, Oct 1, 2013 at 2:04 PM, Christian Hudon wrote: > > What should I do in the future to get code reviews for my pull requests. Should I "just wait and someone will get to it (eventually)", or would it be better to ping people a bit more explicitly? > > > I'd ping someone on IRC or the ML as our response time on pull requests could probably be better =] Thanks for the pull request! I echo Philip on this, since a lot of work happens off the BitBucket interface a gentle reminder to have a look is useful for everyone involved! > > FYI you (or anyone) can manually trigger builds on your own. For future reference (I had no idea where to find this and was lucky enough to find someone to walk me through it at the London sprint) -- head over to http://buildbot.pypy.org/builders/ , find the particular build you're interested in (e.g. http://buildbot.pypy.org/builders/pypy-c-jit-linux-x86-64), fill in a few details and click "Force Build". And then wait a little bit ;-) From drasko.draskovic at gmail.com Mon Oct 7 11:36:02 2013 From: drasko.draskovic at gmail.com (Drasko DRASKOVIC) Date: Mon, 7 Oct 2013 11:36:02 +0200 Subject: [pypy-dev] MIPS port Message-ID: Hi all, what is the state of the MIPS port? Does it exist and is it usable? On the website I can only see ARM ports. Best regards, Drasko From fijall at gmail.com Mon Oct 7 13:51:56 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Mon, 7 Oct 2013 13:51:56 +0200 Subject: [pypy-dev] MIPS port In-Reply-To: References: Message-ID: On Mon, Oct 7, 2013 at 11:36 AM, Drasko DRASKOVIC wrote: > Hi all, > what is the state of the MIPS port? > > Does it exist and is it usable? > > On the website I can only see ARM ports. > > Best regards, > Drasko Hi Drasko Someone was working on a MIPS port for PyPy, but it never was contributed back. There is no official MIPS port in the PyPy repo. Cheers, fijal From cfbolz at gmx.de Wed Oct 9 18:46:49 2013 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 09 Oct 2013 18:46:49 +0200 Subject: [pypy-dev] [pypy-commit] pypy default: be a little more stringent about checking negative numbers early In-Reply-To: <20131009075705.21AC11C02A3@cobra.cs.uni-duesseldorf.de> References: <20131009075705.21AC11C02A3@cobra.cs.uni-duesseldorf.de> Message-ID: <52558879.5030405@gmx.de> Hi Maciej, can you please write some tests for this change? Cheers, Carl Friedrich On 09/10/13 09:57, fijal wrote: > Author: Maciej Fijalkowski > Branch: > Changeset: r67229:c214edb11c30 > Date: 2013-10-09 09:55 +0200 > http://bitbucket.org/pypy/pypy/changeset/c214edb11c30/ > > Log: be a little more stringent about checking negative numbers early > > diff --git a/rpython/annotator/unaryop.py b/rpython/annotator/unaryop.py > --- a/rpython/annotator/unaryop.py > +++ b/rpython/annotator/unaryop.py > @@ -333,12 +333,13 @@ > check_negative_slice(s_start, s_stop) > lst.listdef.resize() > > -def check_negative_slice(s_start, s_stop): > +def check_negative_slice(s_start, s_stop, error="slicing"): > if isinstance(s_start, SomeInteger) and not s_start.nonneg: > - raise AnnotatorError("slicing: not proven to have non-negative start") > + raise AnnotatorError("%s: not proven to have non-negative start" % > + error) > if isinstance(s_stop, SomeInteger) and not s_stop.nonneg and \ > getattr(s_stop, 'const', 0) != -1: > - raise AnnotatorError("slicing: not proven to have non-negative stop") > + raise AnnotatorError("%s: not proven to have non-negative stop" % error) > > > class __extend__(SomeDict): > @@ -448,12 +449,15 @@ > return s_Bool > > def method_find(str, frag, start=None, end=None): > + check_negative_slice(start, end, "find") > return SomeInteger() > > def method_rfind(str, frag, start=None, end=None): > + check_negative_slice(start, end, "rfind") > return SomeInteger() > > def method_count(str, frag, start=None, end=None): > + check_negative_slice(start, end, "count") > return SomeInteger(nonneg=True) > > def method_strip(str, chr): > _______________________________________________ > pypy-commit mailing list > pypy-commit at python.org > https://mail.python.org/mailman/listinfo/pypy-commit > From cfbolz at gmx.de Wed Oct 9 18:48:55 2013 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 09 Oct 2013 18:48:55 +0200 Subject: [pypy-dev] [pypy-commit] pypy fast_cffi_list_init: implement the fast-path for intstrategy and long[] only In-Reply-To: <20131009162428.9A4A51C369D@cobra.cs.uni-duesseldorf.de> References: <20131009162428.9A4A51C369D@cobra.cs.uni-duesseldorf.de> Message-ID: <525588F7.9040309@gmx.de> Hi Anto, just said it on IRC, just so that it doesn't get lost: I think module/_cffi_backend should use the generic interfaces and not touch the internals of listobject.py. it can just call space.listview_int and space.listview_float, they are a no-copy operation on int/float strategy lists. Cheers, Carl Friedrich On 09/10/13 18:24, antocuni wrote: > Author: Antonio Cuni > Branch: fast_cffi_list_init > Changeset: r67250:a8d55ebf78ea > Date: 2013-10-09 18:20 +0200 > http://bitbucket.org/pypy/pypy/changeset/a8d55ebf78ea/ > > Log: implement the fast-path for intstrategy and long[] only > > diff --git a/pypy/module/_cffi_backend/ctypeprim.py b/pypy/module/_cffi_backend/ctypeprim.py > --- a/pypy/module/_cffi_backend/ctypeprim.py > +++ b/pypy/module/_cffi_backend/ctypeprim.py > @@ -85,6 +85,11 @@ > return self.space.wrap(s) > return W_CType.string(self, cdataobj, maxlen) > > + def is_long(self): > + return False > + > + def is_double(self): > + return False > > class W_CTypePrimitiveCharOrUniChar(W_CTypePrimitive): > _attrs_ = [] > @@ -171,6 +176,9 @@ > self.vmin = r_uint(-1) << (sh - 1) > self.vrangemax = (r_uint(1) << sh) - 1 > > + def is_long(self): > + return self.size == rffi.sizeof(lltype.Signed) > + > def cast_to_int(self, cdata): > return self.convert_to_object(cdata) > > @@ -274,6 +282,9 @@ > class W_CTypePrimitiveFloat(W_CTypePrimitive): > _attrs_ = [] > > + def is_double(self): > + return self.size == rffi.sizeof(lltype.Float) > + > def cast(self, w_ob): > space = self.space > if isinstance(w_ob, cdataobj.W_CData): > diff --git a/pypy/module/_cffi_backend/ctypeptr.py b/pypy/module/_cffi_backend/ctypeptr.py > --- a/pypy/module/_cffi_backend/ctypeptr.py > +++ b/pypy/module/_cffi_backend/ctypeptr.py > @@ -58,19 +58,44 @@ > value = rffi.cast(rffi.CCHARP, value) > return cdataobj.W_CData(space, value, self) > > + def _convert_array_from_list_strategy_maybe(self, cdata, w_ob): > + from rpython.rlib.rarray import copy_list_to_raw_array > + from pypy.objspace.std.listobject import W_ListObject, IntegerListStrategy > + if not isinstance(w_ob, W_ListObject): > + return False > + # > + int_stragegy = self.space.fromcache(IntegerListStrategy) > + > + if w_ob.strategy is int_stragegy and self.ctitem.is_long(): > + int_list = w_ob.strategy.unerase(w_ob.lstorage) > + cdata = rffi.cast(rffi.LONGP, cdata) > + copy_list_to_raw_array(int_list, cdata) > + return True > + > + return False > + > + def _convert_array_from_listview(self, cdata, w_ob): > + space = self.space > + lst_w = space.listview(w_ob) > + if self.length >= 0 and len(lst_w) > self.length: > + raise operationerrfmt(space.w_IndexError, > + "too many initializers for '%s' (got %d)", > + self.name, len(lst_w)) > + ctitem = self.ctitem > + for i in range(len(lst_w)): > + ctitem.convert_from_object(cdata, lst_w[i]) > + cdata = rffi.ptradd(cdata, ctitem.size) > + > def convert_array_from_object(self, cdata, w_ob): > space = self.space > + if self._convert_array_from_list_strategy_maybe(cdata, w_ob): > + # the fast path worked, we are done now > + return > + # > + # continue with the slow path > if (space.isinstance_w(w_ob, space.w_list) or > space.isinstance_w(w_ob, space.w_tuple)): > - lst_w = space.listview(w_ob) > - if self.length >= 0 and len(lst_w) > self.length: > - raise operationerrfmt(space.w_IndexError, > - "too many initializers for '%s' (got %d)", > - self.name, len(lst_w)) > - ctitem = self.ctitem > - for i in range(len(lst_w)): > - ctitem.convert_from_object(cdata, lst_w[i]) > - cdata = rffi.ptradd(cdata, ctitem.size) > + self._convert_array_from_listview(cdata, w_ob) > elif (self.can_cast_anything or > (self.ctitem.is_primitive_integer and > self.ctitem.size == rffi.sizeof(lltype.Char))): > diff --git a/pypy/objspace/std/listobject.py b/pypy/objspace/std/listobject.py > --- a/pypy/objspace/std/listobject.py > +++ b/pypy/objspace/std/listobject.py > @@ -139,6 +139,8 @@ > > class W_ListObject(W_Root): > > + strategy = None > + > def __init__(self, space, wrappeditems, sizehint=-1): > assert isinstance(wrappeditems, list) > self.space = space > _______________________________________________ > pypy-commit mailing list > pypy-commit at python.org > https://mail.python.org/mailman/listinfo/pypy-commit > From anto.cuni at gmail.com Thu Oct 10 10:56:46 2013 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 10 Oct 2013 10:56:46 +0200 Subject: [pypy-dev] [pypy-commit] pypy fast_cffi_list_init: implement the fast-path for intstrategy and long[] only In-Reply-To: <525588F7.9040309@gmx.de> References: <20131009162428.9A4A51C369D@cobra.cs.uni-duesseldorf.de> <525588F7.9040309@gmx.de> Message-ID: <52566BCE.5030204@gmail.com> Hi, On 09/10/13 18:48, Carl Friedrich Bolz wrote: > Hi Anto, > > just said it on IRC, just so that it doesn't get lost: > > I think module/_cffi_backend should use the generic interfaces and not > touch the internals of listobject.py. it can just call > space.listview_int and space.listview_float, they are a no-copy > operation on int/float strategy lists. indeed, that's a very good idea. I should have thought of it :) From matti.picus at gmail.com Tue Oct 15 06:15:27 2013 From: matti.picus at gmail.com (Matti Picus) Date: Tue, 15 Oct 2013 07:15:27 +0300 Subject: [pypy-dev] progress with numpy and removal of numpy.py Message-ID: <525CC15F.70103@gmail.com> I have been working to get numfocus benchmarks written for numpy to run on numpypy and pypy https://github.com/numfocus/python-benchmarks The biggest obstacle is actually the need for matplotlib, which is used internally to produce pretty comparisons between different benchmarks. Luckily, Stefan H. Muller did much of what is needed to use matplotlib (non-interactively) with pypy, I have taken over his work and it now lives in two git repos: https://github.com/mattip/numpy on the pypy-hack branch, and https://github.com/mattip/matplotlib master branch. These work, download a nightly pypy, virtualenv, "setup.py install", but there is some missing low-lying-fruit kind of functionality to run the benchmarks, again, in matplotlib. That is the good news, the bad news is that in a perhaps hasty move, I removed lib_pypy/numpy.py since it complicates the numpy install. I fixed tests (one build too late, sorry for the failures), but now people who were used to a nightly pypy being capable of "import numpy" will be disappointed. I did ask on IRC before the change, and got two positive votes, but feel a need for wider consensus. So should we leave the file out, and hopefully encourage people to start trying the pypy-hack branch, or put it back? In an unrelated-but-related issue, should the github repo of numpy be moved to a more pypy-like place (bitbucket.org/pypy)? Matti From yury at shurup.com Tue Oct 15 08:53:48 2013 From: yury at shurup.com (Yury V. Zaytsev) Date: Tue, 15 Oct 2013 08:53:48 +0200 Subject: [pypy-dev] progress with numpy and removal of numpy.py In-Reply-To: <525CC15F.70103@gmail.com> References: <525CC15F.70103@gmail.com> Message-ID: <1381820028.2861.8.camel@newpride> On Tue, 2013-10-15 at 07:15 +0300, Matti Picus wrote: > > I did ask on IRC before the change, and got two positive votes, but > feel a need for wider consensus. Does that mean that now one only can import NumPyPy as follows import numpypy instead of import numpy ? If yes, what is going to be the way forward? I mean, is the plan such that eventually, you should be able to install PyPy that bundles NumPyPy and then NumPy (when the patches will be accepted upstream, or maybe a PyPy-compatible fork will be released on pypi)? If your github repository works similarly or better than `import numpy` as implemented before, I see no problem with it and will just adapt my scripts; I only feel that this should be better documented, or advertised more prominently. For example, you might consider posting a progress report on the blog which explains the changes and tells how to deal with them... -- Sincerely yours, Yury V. Zaytsev From romain.py at gmail.com Tue Oct 15 12:18:48 2013 From: romain.py at gmail.com (Romain Guillebert) Date: Tue, 15 Oct 2013 12:18:48 +0200 Subject: [pypy-dev] progress with numpy and removal of numpy.py In-Reply-To: <525CC15F.70103@gmail.com> References: <525CC15F.70103@gmail.com> Message-ID: Hi Matti I think we should remove numpy.py and package the pypy-hack branch (even if it's not stable yet) so that people can use it directly. Cheers Romain On Tue, Oct 15, 2013 at 6:15 AM, Matti Picus wrote: > I have been working to get numfocus benchmarks written for numpy to run on > numpypy and pypy https://github.com/numfocus/python-benchmarks > > The biggest obstacle is actually the need for matplotlib, which is used > internally to produce pretty comparisons between different benchmarks. > Luckily, Stefan H. Muller did much of what is needed to use matplotlib > (non-interactively) with pypy, I have taken over his work and it now lives > in two git repos: https://github.com/mattip/numpy on the pypy-hack branch, > and https://github.com/mattip/matplotlib master branch. These work, download > a nightly pypy, > virtualenv, "setup.py install", but there is some missing low-lying-fruit > kind of functionality to run the benchmarks, again, in matplotlib. > > That is the good news, the bad news is that in a perhaps hasty move, I > removed lib_pypy/numpy.py since it complicates the numpy install. I fixed > tests (one build too late, sorry for the failures), but now people who were > used to a nightly pypy being capable of "import numpy" will be disappointed. > > I did ask on IRC before the change, and got two positive votes, but feel a > need for wider consensus. So should we leave the file out, and hopefully > encourage people to start trying the pypy-hack branch, or put it back? In an > unrelated-but-related issue, should the github repo of numpy be moved to a > more pypy-like place (bitbucket.org/pypy)? > > Matti > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From matti.picus at gmail.com Tue Oct 15 12:45:24 2013 From: matti.picus at gmail.com (matti picus) Date: Tue, 15 Oct 2013 13:45:24 +0300 Subject: [pypy-dev] progress with numpy and removal of numpy.py In-Reply-To: References: <525CC15F.70103@gmail.com> Message-ID: and what about cloning the numpy repo into bitbucket/pypy to make it more of a "pypy owned" thing? Matti On Tue, Oct 15, 2013 at 1:18 PM, Romain Guillebert wrote: > Hi Matti > > I think we should remove numpy.py and package the pypy-hack branch > (even if it's not stable yet) so that people can use it directly. > > Cheers > Romain > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricardohenrylee at gmail.com Tue Oct 15 17:50:22 2013 From: ricardohenrylee at gmail.com (Richard H Lee) Date: Tue, 15 Oct 2013 16:50:22 +0100 Subject: [pypy-dev] Cygwin pypy patches (cpyext module) Message-ID: <525D643E.1090907@gmail.com> Hi Uwe, I came across your tux.org page on how to build pypy on cygwin. I was wondering if you made any patches against the cpyext module to get it to build on cygwin? Richard From uwe_f_mayer at yahoo.com Tue Oct 15 18:28:06 2013 From: uwe_f_mayer at yahoo.com (Uwe F. Mayer) Date: Tue, 15 Oct 2013 09:28:06 -0700 (PDT) Subject: [pypy-dev] Cygwin pypy patches (cpyext module) In-Reply-To: <525D643E.1090907@gmail.com> References: <525D643E.1090907@gmail.com> Message-ID: <1381854486.72133.YahooMailNeo@web162803.mail.bf1.yahoo.com> Richard, sorry no, I have not worked on that. -- Uwe >________________________________ > From: Richard H Lee >To: uwe_f_mayer at yahoo.com >Cc: pypy-dev at python.org >Sent: Tuesday, October 15, 2013 8:50 AM >Subject: Cygwin pypy patches (cpyext module) > > >Hi Uwe, > >I came across your tux.org page on how to build pypy on cygwin. > >I was wondering if you made any patches against the cpyext module to get >it to build on cygwin? > > >Richard > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anto.cuni at gmail.com Thu Oct 17 10:59:55 2013 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 17 Oct 2013 10:59:55 +0200 Subject: [pypy-dev] progress with numpy and removal of numpy.py In-Reply-To: References: <525CC15F.70103@gmail.com> Message-ID: <525FA70B.6010009@gmail.com> Hi, On 15/10/13 12:45, matti picus wrote: > and what about cloning the numpy repo into bitbucket/pypy to make it more of a > "pypy owned" thing? I think it's a good idea. I propose the following: 1) we move your repo to bitbucket/pypy/numpypy 2) we package numpypy, so that people can just do "pip install numpypy" 3) once numpypy is installed, we no longer require the ugly "import numpypy"; a simple "import numpy" will just work. 4) for some time at least, we distribute a numpypy.py so that when imported it prints an error message which explain how to get the newer numpypy What do you think? From claudia.steiner at multimediaoutreach.com Mon Oct 21 14:21:37 2013 From: claudia.steiner at multimediaoutreach.com (Claudia Steiner) Date: Mon, 21 Oct 2013 15:21:37 +0300 Subject: [pypy-dev] Advertising at your site http://playulty.com Message-ID: <100003755198482358712959@guyz-64-pc> An HTML attachment was scrubbed... URL: From andrewfr_ice at yahoo.com Tue Oct 22 23:10:00 2013 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Tue, 22 Oct 2013 14:10:00 -0700 (PDT) Subject: [pypy-dev] pypy-stm and lock-free data structures Message-ID: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> Hi Folks: I have been reading a few papers on lock-free data structures (and reading some posts on Golang Nuts). I was thinking about writing new a version of stackless.py with Go's select. However I would try to use lock free data structures and techniques in the channel implementation. My questions: 1) Would I be using the pypy-stm branch? (because there is no GIL) 2) Would I write some of this in RPython? (I am assuming yes) 3) Do I have access to low-level atomic operations (I don't think the high-level transaction module gives me access)? 4) Does stackless features work with pypy-stm (I haven't yet been successful in compiling a pypy-stm branch so I can't test this out myself). Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Thu Oct 24 10:38:02 2013 From: arigo at tunes.org (Armin Rigo) Date: Thu, 24 Oct 2013 10:38:02 +0200 Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: Hi Andrew, On Tue, Oct 22, 2013 at 11:10 PM, Andrew Francis wrote: > 1) Would I be using the pypy-stm branch? (because there is no GIL) > 2) Would I write some of this in RPython? (I am assuming yes) The pypy-stm branch has no GIL in the implementation, but its behavior is equivalent to the GIL. I can't answer your questions without knowing exactly what you're planning. Sorry if I forgot, I know you mentioned it several times here. I still have a hard time figuring out the real goal, rather than implementation details. For example, "I want to implement what is described in paper X about language Y" is often unrelated to "can I use pypy-stm's GIL approach". > 3) Do I have access to low-level atomic operations (I don't think the > high-level transaction module gives me access)? rpython.rlib.atomic_ops. Note however that the core of pypy-stm is directly implemented in C, not in RPython. > 4) Does stackless features work with pypy-stm (I haven't yet been successful > in compiling a pypy-stm branch so I can't test this out myself). I guess not right now, but that should be easy if someone cares. A bient?t, Armin. From andrewfr_ice at yahoo.com Thu Oct 24 20:27:05 2013 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Thu, 24 Oct 2013 11:27:05 -0700 (PDT) Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> Hi Armin: On Thursday, October 24, 2013 4:38 AM, Armin Rigo wrote: > ?I can't answer your questions without >knowing exactly what you're planning.? Sorry if I forgot, I know you >mentioned it several times here.? I still have a hard time figuring >out the real goal, rather than implementation details.? For example, >"I want to implement what is described in paper X about language Y" is >often unrelated to "can I use pypy-stm's GIL approach". The University of Chicago/Microsoft Research paper "Scalable Join Patterns" (http://www.mpi-sws.org/~turon/scalable-joins.pdf) outlines an approach for implementing join patterns using essentially lock-free bags, optimistic locking and techniques for guaranteeing liveness. For the sake of this discussion, ignore the term "join pattern" : substitute it with "channel." A bit more context - I have looked at the C code for the Go channel implementation and said "wow, there is a lot of locking there"! I also have been following conversations concerning race conditions and the race detector (based on thread sanitizer) and asking myself: pypy-stm could do this. For some experiments, I want pypy-stm for two reasons: 1) There is no GIL so I can compare the results of say a lock-free queue to say, Python thread-safe queues (I am not sure of their utility in pypy-stm). 2) I may need some of pypy-stm's functionality. Is there anything wrong with this reasoning? >rpython.rlib.atomic_ops.?? Yes. >Note however that the core of pypy-stm is?directly implemented in C, not >in RPython. Okay I'll look at the code. A little while back I looked at RSTM. >I guess not right now, but that should be easy if someone cares. I care. What do I need to look at? Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From njh at njhurst.com Fri Oct 25 00:22:56 2013 From: njh at njhurst.com (Nathan Hurst) Date: Fri, 25 Oct 2013 09:22:56 +1100 Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: <20131024222256.GB18706@ajhurst.org> On Thu, Oct 24, 2013 at 11:27:05AM -0700, Andrew Francis wrote: > The University of Chicago/Microsoft Research paper "Scalable Join > Patterns" (http://www.mpi-sws.org/~turon/scalable-joins.pdf) > outlines an approach for implementing join patterns using > essentially lock-free bags, optimistic locking and techniques for > guaranteeing liveness. For the sake of this discussion, ignore the > term "join pattern" : substitute it with "channel." > > A bit more context - I have looked at the C code for the Go channel > implementation and said "wow, there is a lot of locking there"! I > also have been following conversations concerning race conditions > and the race detector (based on thread sanitizer) and asking myself: > pypy-stm could do this. Thanks for the paper reference, very interesting. Is it worth implementing the api first with a naive implementation with some examples of use before heading into the details of optimised implementation? (clearly it won't prove performance value, but at least half of the merit of that paper is the clean declarative style and precise semantics of the join calculus) njh From arigo at tunes.org Fri Oct 25 10:35:49 2013 From: arigo at tunes.org (Armin Rigo) Date: Fri, 25 Oct 2013 10:35:49 +0200 Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: Hi Andrew, On Thu, Oct 24, 2013 at 8:27 PM, Andrew Francis wrote: > A bit more context - I have looked at the C code for the Go channel > implementation and said "wow, there is a lot of locking there"! I also have > been following conversations concerning race conditions and the race > detector (based on thread sanitizer) and asking myself: pypy-stm could do > this. Precisely my point. You're coming with paper X describing Y and thinking "pypy-stm could do this", whereas I don't see any obvious connection between the two. To repeat myself, pypy-stm has no GIL internally, but exposes to the user semantics that are the same as a GIL Python. To be on the clear side, you could use CPython and write C, with GIL-releasing code around the complicated lock-free algorithm; but the point of the lock-free algorithm is to be very fast, negating the advantage of releasing the GIL in the first place. Is that correct? Then if you want to integrate it inside pypy-stm instead, it's more complicated than that: we don't want an algorithm (say a simple lock-free queue) without some behavior that is tailored to the stm world. For example, if you implement only a queue, then: - pops will appear out of order if they are done in some order but the transactions commit in a different orders; - pops will be lost if they are done by a transaction that aborts; - pushes must be at least delayed until the transaction commits, otherwise we see bogus data from other threads. So you need to think about it more deeply than just "pypy-stm is cool, let's use it for X". You can't just think about pypy-stm as a lock-free Python, because that's only half the truth. PyPy-stm is all about a special use of lock-free algorithms in order to give the user the illusion of the GIL. I would say that it's not really the correct ground to experiment with lock-free structures in general --- it's a place where we can experiment with custom adaptations of some lock-free structures, done towards a specific goal that makes sense for the user of pypy-stm. A bient?t, Armin. From tlangevin at shaw.ca Fri Oct 25 19:20:44 2013 From: tlangevin at shaw.ca (Trevor Langevin) Date: Fri, 25 Oct 2013 10:20:44 -0700 Subject: [pypy-dev] error message with windows Message-ID: <526AA86C.1050307@shaw.ca> Hello, I'm learning 3D printing, and to use Skeinforge it was recommended to download your pypy program to use on a windows machine. I get an error message while trying to unzip you program "Please insert the last disk of the multi-disk set?" Where do I find this disk, its not included in the installer package? Trev I could not log on to your bug tracker site, firefox would not let me view the page as its listed as untrusted. From andrewfr_ice at yahoo.com Fri Oct 25 19:26:40 2013 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Fri, 25 Oct 2013 10:26:40 -0700 (PDT) Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: <1382722000.258.YahooMailNeo@web140701.mail.bf1.yahoo.com> Hi Armin: On Friday, October 25, 2013 4:36 AM, Armin Rigo wrote: >Precisely my point.? You're coming with paper X describing Y and >thinking "pypy-stm could do this", whereas I don't see any obvious >connection between the two.? To repeat myself, pypy-stm has no GIL >internally, but exposes to the user semantics that are the same as a >GIL Python. I understand your point. In regards to the Go race detector, I am interested in the collision detection aspects. For now this is secondary. The thing I am mostly interested in is avoiding fine grained locking and everything that entails.? >To be on the clear side, you could use CPython and write C, with >GIL-releasing code around the complicated lock-free algorithm; but the >point of the lock-free algorithm is to be very fast, negating the >advantage of releasing the GIL in the first place.? Is that correct? Yes. I want a Python environment where I can test lock free algorithms without the GIL getting in the way of analysis. I believe there are tricks that essentially turn off the GIL but I would prefer it not being there. Maybe my reasoning is wrong? >Then if you want to integrate it inside pypy-stm instead, it's more >complicated than that: we don't want an algorithm (say a simple >lock-free queue) without some behavior that is tailored to the stm >world.? For example, if you implement only a queue, then: - pops will >appear out of order if they are done in some order but the >transactions commit in a different orders; - pops will be lost if they >are done by a transaction that aborts; - pushes must be at least >delayed until the transaction commits, otherwise we see bogus data >from other threads. Yes I can see these problems occurring if the lock-free implementation and the STM are not aware of one-another. However the underlying assumption is that both the STM/transaction model and Stackless API (i.e., send(), receive()) are available to the application programmer. And underlying Stackless and STM machinery are orthogonal (for example, Haskell STM uses its light-weight thread scheduler) . It may just so happen that the lower level pypy-stm machinery is being used to implement the Stackless API.?It is way too early to say. And I don't know enough. >So you need to think about it more deeply than just "pypy-stm is cool, >let's use it for X".? You can't just think about pypy-stm as a >lock-free Python, because that's only half the truth.? PyPy-stm is all >about a special use of lock-free algorithms in order to give the user >the illusion of the GIL.? I would say that it's not really the correct >ground to experiment with lock-free structures in general --- it's a >place where we can experiment with custom adaptations of some >lock-free structures, done towards a specific goal that makes sense >for the user of pypy-stm. I understand your point.? I tend to look at things from the perspective of a Stackless Python user. I also want to be a pypy-stm user too! I am going under the assumption that I can use STM feature in much the same way I use the PyPy stackless.py implementation: as an erector set for experimenting with new features.? In the bigger scheme of things, I am buying an argument made in the Scalable Join Pattern paper. For this discussion, treat join pattern as synonymous with channels/message passing. The argument is that although efficient implementation of join patterns and the STM both involve a transactional aspect, STM is a more general solution, hence harder to optimise.? Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrewfr_ice at yahoo.com Fri Oct 25 19:27:14 2013 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Fri, 25 Oct 2013 10:27:14 -0700 (PDT) Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> Message-ID: <1382722034.50222.YahooMailNeo@web140705.mail.bf1.yahoo.com> Hi Armin: On Friday, October 25, 2013 4:36 AM, Armin Rigo wrote: >Precisely my point.? You're coming with paper X describing Y and >thinking "pypy-stm could do this", whereas I don't see any obvious >connection between the two.? To repeat myself, pypy-stm has no GIL >internally, but exposes to the user semantics that are the same as a >GIL Python. I understand your point. In regards to the Go race detector, I am interested in the collision detection aspects. For now this is secondary. The thing I am mostly interested in is avoiding fine grained locking and everything that entails.? >To be on the clear side, you could use CPython and write C, with >GIL-releasing code around the complicated lock-free algorithm; but the >point of the lock-free algorithm is to be very fast, negating the >advantage of releasing the GIL in the first place.? Is that correct? Yes. I want a Python environment where I can test lock free algorithms without the GIL getting in the way of analysis. I believe there are tricks that essentially turn off the GIL but I would prefer it not being there. Maybe my reasoning is wrong? >Then if you want to integrate it inside pypy-stm instead, it's more >complicated than that: we don't want an algorithm (say a simple >lock-free queue) without some behavior that is tailored to the stm >world.? For example, if you implement only a queue, then: - pops will >appear out of order if they are done in some order but the >transactions commit in a different orders; - pops will be lost if they >are done by a transaction that aborts; - pushes must be at least >delayed until the transaction commits, otherwise we see bogus data >from other threads. Yes I can see these problems occurring if the lock-free implementation and the STM are not aware of one-another. However the underlying assumption is that both the STM/transaction model and Stackless API (i.e., send(), receive()) are available to the application programmer. And underlying Stackless and STM machinery are orthogonal (for example, Haskell STM uses its light-weight thread scheduler) . It may just so happen that the lower level pypy-stm machinery is being used to implement the Stackless API.?It is way too early to say. And I don't know enough. >So you need to think about it more deeply than just "pypy-stm is cool, >let's use it for X".? You can't just think about pypy-stm as a >lock-free Python, because that's only half the truth.? PyPy-stm is all >about a special use of lock-free algorithms in order to give the user >the illusion of the GIL.? I would say that it's not really the correct >ground to experiment with lock-free structures in general --- it's a >place where we can experiment with custom adaptations of some >lock-free structures, done towards a specific goal that makes sense >for the user of pypy-stm. I understand your point.? I tend to look at things from the perspective of a Stackless Python user. I also want to be a pypy-stm user too! I am going under the assumption that I can use STM feature in much the same way I use the PyPy stackless.py implementation: as an erector set for experimenting with new features.? In the bigger scheme of things, I am buying an argument made in the Scalable Join Pattern paper. For this discussion, treat join pattern as synonymous with channels/message passing. The argument is that although efficient implementation of join patterns and the STM both involve a transactional aspect, STM is a more general solution, hence harder to optimise.? Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrewfr_ice at yahoo.com Fri Oct 25 19:30:57 2013 From: andrewfr_ice at yahoo.com (Andrew Francis) Date: Fri, 25 Oct 2013 10:30:57 -0700 (PDT) Subject: [pypy-dev] pypy-stm and lock-free data structures In-Reply-To: <20131024222256.GB18706@ajhurst.org> References: <1382476200.48161.YahooMailNeo@web140703.mail.bf1.yahoo.com> <1382639225.3031.YahooMailNeo@web140703.mail.bf1.yahoo.com> <20131024222256.GB18706@ajhurst.org> Message-ID: <1382722257.87567.YahooMailNeo@web140706.mail.bf1.yahoo.com> Hi Nathan: On Thursday, October 24, 2013 6:21 PM, Nathan Hurst wrote: >Thanks for the paper reference, very interesting.? It is a really cool paper. I have spoken to the authors. > Is it worth?implementing the api first with a naive implementation with >some Not really. the Join Patterns API and the lock-free algorithms are different problems. However little while back, I wrote an initial join pattern implementation based on a modified version of stackless.py. It works but the API is bad and the implementation has a much to be desired. I have been?I have been writing a new version with a much simpler API that can run onto of C based Stackless Python, CPython with greenlets and PyPy. If you want more details, we can take this discussion offline. But some details are at?http://andrewfr.wordpress.com/ Cheers, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From rami.chowdhury at gmail.com Sat Oct 26 00:15:37 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Fri, 25 Oct 2013 23:15:37 +0100 Subject: [pypy-dev] error message with windows In-Reply-To: <526AA86C.1050307@shaw.ca> References: <526AA86C.1050307@shaw.ca> Message-ID: <3EE6B7116D8748ED9352DA34A6652652@gmail.com> Hi Trevor, On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: > I get an error message while trying to unzip you program "Please insert > the last disk of the multi-disk set?" > > Where do I find this disk, its not included in the installer package? Can you please clarify which installer package you downloaded? Thanks Rami From kev at inburke.com Sat Oct 26 07:18:23 2013 From: kev at inburke.com (Kevin Burke) Date: Fri, 25 Oct 2013 22:18:23 -0700 Subject: [pypy-dev] A few notes trying to compile pypy Message-ID: Hey everyone, I was trying to compile pypy and made the following mistakes: - On this page: http://pypy.org/download.html I couldn't figure out whether I was supposed to run all three of these commands: pypy ../../rpython/bin/rpython -Ojit targetpypystandalone # get the JIT version pypy ../../rpython/bin/rpython -O2 targetpypystandalone # get the no-jit version pypy ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone # get the sandbox version or just one. - Once I got it installled I didn't realize there was an additional step after pypy-c was created. - I tried running python package.py -h and that didn't do what I expected (show help dialog) - I tried running package.py with the right options and it hung with this message (on both Mac and Centos): sll_os.ll_os_getenv(sPYPY_GENERATIONGC_NURSERY The only Google search results for this are an IRC channel from 2 years ago. It turned out I didn't want the Sandbox option, I think. - I tried compiling outside the sandbox and got this error: raise NoTTY("Cannot start the debugger when stdout is captured.") It turns out this is a red herring and the real error was way above that: error: ffi.h: No such file or directory It would be nice if these were more obvious.. of course I expect to run into issues but the errors could be more clear when they are. Continuing the install process now, will post updates here as I have them. I am happy to help try and fix these where I can, if you can point me in the right direction - I have a fair amount of experience working with new users at Twilio. I have also made exactly one commit to Pypy source code: https://bitbucket.org/pypy/pypy/commits/665baf750859/ Pypy is a lovely project and I'd love to try and help make this process easier. Best, Kevin -- Kevin Burke | Twilio phone: 925.271.7005 | kev.inburke.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From fijall at gmail.com Sat Oct 26 10:25:48 2013 From: fijall at gmail.com (Maciej Fijalkowski) Date: Sat, 26 Oct 2013 10:25:48 +0200 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Hi Kevin, thanks for the feedback! Do you have maybe an idea how to improve the docs? about libffi.h - Apple removed /usr/include and moved it somewhere else, no clue where. I also don't have access to Mavericks so I can't fix it. On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: > Hey everyone, > I was trying to compile pypy and made the following mistakes: > > - On this page: http://pypy.org/download.html I couldn't figure out whether > I was supposed to run all three of these commands: > > pypy ../../rpython/bin/rpython -Ojit targetpypystandalone # get > the JIT version > pypy ../../rpython/bin/rpython -O2 targetpypystandalone # get > the no-jit version > pypy ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone # get > the sandbox version > > or just one. > > - Once I got it installled I didn't realize there was an additional step > after pypy-c was created. > > - I tried running python package.py -h and that didn't do what I expected > (show help dialog) > > - I tried running package.py with the right options and it hung with this > message (on both Mac and Centos): > > sll_os.ll_os_getenv(sPYPY_GENERATIONGC_NURSERY > > The only Google search results for this are an IRC channel from 2 years ago. > It turned out I didn't want the Sandbox option, I think. > > - I tried compiling outside the sandbox and got this error: > > raise NoTTY("Cannot start the debugger when stdout is captured.") > > It turns out this is a red herring and the real error was way above that: > > error: ffi.h: No such file or directory > > It would be nice if these were more obvious.. of course I expect to run into > issues but the errors could be more clear when they are. > > Continuing the install process now, will post updates here as I have them. > > I am happy to help try and fix these where I can, if you can point me in the > right direction - I have a fair amount of experience working with new users > at Twilio. I have also made exactly one commit to Pypy source code: > https://bitbucket.org/pypy/pypy/commits/665baf750859/ Pypy is a lovely > project and I'd love to try and help make this process easier. > > Best, > Kevin > > -- > Kevin Burke | Twilio > phone: 925.271.7005 | kev.inburke.com > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > From arigo at tunes.org Sat Oct 26 16:35:45 2013 From: arigo at tunes.org (Armin Rigo) Date: Sat, 26 Oct 2013 16:35:45 +0200 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Hi, On Sat, Oct 26, 2013 at 10:25 AM, Maciej Fijalkowski wrote: > about libffi.h - Apple removed /usr/include and moved it somewhere > else, no clue where. I also don't have access to Mavericks so I can't > fix it. Ooooh. Traditionally, "libffi.h" was put in unexpectedly varying directories on every Linux distribution, just to make sure there was no reliable way to find it, but that was resolved some time ago with pkg-config. Apple's move is on a completely different level: killing the whole /usr/include is a perfect way to break compilation for every open source project under the sun. I'm impressed. A bient?t, Armin. From kev at inburke.com Sat Oct 26 16:16:57 2013 From: kev at inburke.com (Kevin Burke) Date: Sat, 26 Oct 2013 07:16:57 -0700 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: A few more problems I ran into: - package.py lists these options: package.py root-pypy-dir [--nostrip] [--without-tk] [name-of-archive] [name-of-pypy-c] [destination-for-tarball] [pypy-c-path] Running in this order package.py ../../.. --without-tk means the --without-tk option doesn't get registered properly. Currently I am running into this error: error: 'SQLITE_DROP_VTABLE' undeclared (first use in this function) My version of sqlite3 is too old, but I can't upgrade the system sqlite3, so I'm trying to link a different one using LD_LIBRARY_PATH like this LD_LIBRARY_PATH=/path/to/sqlite3/sqlite-autoconf-3080100:$LD_LIBRARY_PATH /path/to/pypy-2.1-src/pypy/goal/pypy-c -c 'import _sqlite3' but still getting the same error, so not sure what to do at this point. Here are a few things I could add to the docs: 1. explain what each of the targetpypystandalone options does and that they are exclusive, not consecutive 2. add a numbered list for the "Building from source" section to emphasize you're supposed to run each number in the process to build a pypy 3. at the end of each numbered step, explain the output, eg "This will produce a binary called pypy-c that you can then use to release a pypy binary" 4. wrap subprocess.check_call("pypy-c -c 'import _sqlite3'") in a try/catch and add output 5. use argparse instead of sys.argv for package.py .. ? I think the mimimum python version now is 2.6 so this should be supported. -- Kevin Burke | Twilio phone: 925.271.7005 | kev.inburke.com On Fri, Oct 25, 2013 at 10:18 PM, Kevin Burke wrote: > Hey everyone, > I was trying to compile pypy and made the following mistakes: > > - On this page: http://pypy.org/download.html I couldn't figure out > whether I was supposed to run all three of these commands: > > pypy ../../rpython/bin/rpython -Ojit targetpypystandalone # get > the JIT version > pypy ../../rpython/bin/rpython -O2 targetpypystandalone # get > the no-jit version > pypy ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone # get > the sandbox version > > or just one. > > - Once I got it installled I didn't realize there was an additional step > after pypy-c was created. > > - I tried running python package.py -h and that didn't do what I expected > (show help dialog) > > - I tried running package.py with the right options and it hung with this > message (on both Mac and Centos): > > sll_os.ll_os_getenv(sPYPY_GENERATIONGC_NURSERY > > The only Google search results for this are an IRC channel from 2 years > ago. It turned out I didn't want the Sandbox option, I think. > > - I tried compiling outside the sandbox and got this error: > > raise NoTTY("Cannot start the debugger when stdout is captured.") > > It turns out this is a red herring and the real error was way above that: > > error: ffi.h: No such file or directory > > It would be nice if these were more obvious.. of course I expect to run > into issues but the errors could be more clear when they are. > > Continuing the install process now, will post updates here as I have them. > > I am happy to help try and fix these where I can, if you can point me in > the right direction - I have a fair amount of experience working with new > users at Twilio. I have also made exactly one commit to Pypy source code: > https://bitbucket.org/pypy/pypy/commits/665baf750859/ Pypy is a lovely > project and I'd love to try and help make this process easier. > > Best, > Kevin > > -- > Kevin Burke | Twilio > phone: 925.271.7005 | kev.inburke.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sat Oct 26 16:46:44 2013 From: arigo at tunes.org (Armin Rigo) Date: Sat, 26 Oct 2013 16:46:44 +0200 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Hi Kevin, Thanks for this report ! A quick note though: On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: > - Once I got it installled I didn't realize there was an additional step > after pypy-c was created. Note that this step, "package.py", is not required to run the pypy-c executable. It's merely a tool that packs all files needed for system-wide installation. It also prebuilds a few cffi modules. The alternative is to simply use the pypy-c where it is (pypy/goal/pypy-c) instead, if you don't need it to be installed system-wide. Symlinks pointing to pypy/goal/pypy-c work fine. A bient?t, Armin. From kev at inburke.com Sat Oct 26 16:53:07 2013 From: kev at inburke.com (Kevin Burke) Date: Sat, 26 Oct 2013 07:53:07 -0700 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Hi Maciej, I should note - I'm on Mountain Lion so libffi.h is not the issue at the moment - currently trying to compile this on Centos get the #include in _sqlite3.py to find my custom sqlite3.h and not the one in /usr/lib. Here is the compile script I am using: https://gist.github.com/kevinburke/7170273 -- Kevin Burke | Twilio phone: 925.271.7005 | kev.inburke.com On Sat, Oct 26, 2013 at 7:46 AM, Armin Rigo wrote: > Hi Kevin, > > Thanks for this report ! A quick note though: > > On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: > > - Once I got it installled I didn't realize there was an additional step > > after pypy-c was created. > > Note that this step, "package.py", is not required to run the pypy-c > executable. It's merely a tool that packs all files needed for > system-wide installation. It also prebuilds a few cffi modules. The > alternative is to simply use the pypy-c where it is (pypy/goal/pypy-c) > instead, if you don't need it to be installed system-wide. Symlinks > pointing to pypy/goal/pypy-c work fine. > > > A bient?t, > > Armin. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From taavi.burns at gmail.com Sat Oct 26 17:14:58 2013 From: taavi.burns at gmail.com (Taavi Burns) Date: Sat, 26 Oct 2013 11:14:58 -0400 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: On Mavericks: $ sudo find / -type f -name \*ffi.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/ffi/ffi.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/ffi/ffi.h /Users/taavi/src/pypy/rpython/translator/c/src/libffi_msvc/ffi.h /usr/local/Cellar/libffi/3.0.13/lib/libffi-3.0.13/include/ffi.h Some quick googling indicates that yes, /usr/include moved into the platform SDK inside the XCode application. Sounds like one can restore it by installing the command-line tools (from http://stackoverflow.com/questions/19531262/cant-phpize-or-configure-an-extension-in-os-x-10-9-mavericks ): xcode-select --install Now I have a /usr/include again! $ ls /usr/include AssertMacros.h cache.h ? I haven't tried building pypy since the upgrade though. On Sat, Oct 26, 2013 at 4:25 AM, Maciej Fijalkowski wrote: > Hi Kevin, thanks for the feedback! > > Do you have maybe an idea how to improve the docs? > > about libffi.h - Apple removed /usr/include and moved it somewhere > else, no clue where. I also don't have access to Mavericks so I can't > fix it. > > On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: >> Hey everyone, >> I was trying to compile pypy and made the following mistakes: >> >> - On this page: http://pypy.org/download.html I couldn't figure out whether >> I was supposed to run all three of these commands: >> >> pypy ../../rpython/bin/rpython -Ojit targetpypystandalone # get >> the JIT version >> pypy ../../rpython/bin/rpython -O2 targetpypystandalone # get >> the no-jit version >> pypy ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone # get >> the sandbox version >> >> or just one. >> >> - Once I got it installled I didn't realize there was an additional step >> after pypy-c was created. >> >> - I tried running python package.py -h and that didn't do what I expected >> (show help dialog) >> >> - I tried running package.py with the right options and it hung with this >> message (on both Mac and Centos): >> >> sll_os.ll_os_getenv(sPYPY_GENERATIONGC_NURSERY >> >> The only Google search results for this are an IRC channel from 2 years ago. >> It turned out I didn't want the Sandbox option, I think. >> >> - I tried compiling outside the sandbox and got this error: >> >> raise NoTTY("Cannot start the debugger when stdout is captured.") >> >> It turns out this is a red herring and the real error was way above that: >> >> error: ffi.h: No such file or directory >> >> It would be nice if these were more obvious.. of course I expect to run into >> issues but the errors could be more clear when they are. >> >> Continuing the install process now, will post updates here as I have them. >> >> I am happy to help try and fix these where I can, if you can point me in the >> right direction - I have a fair amount of experience working with new users >> at Twilio. I have also made exactly one commit to Pypy source code: >> https://bitbucket.org/pypy/pypy/commits/665baf750859/ Pypy is a lovely >> project and I'd love to try and help make this process easier. >> >> Best, >> Kevin >> >> -- >> Kevin Burke | Twilio >> phone: 925.271.7005 | kev.inburke.com >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -- taa /*eof*/ From alex.gaynor at gmail.com Sat Oct 26 17:25:11 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 26 Oct 2013 08:25:11 -0700 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Yup, if you run `xcode-select --install` everything fixes itself. Alex On Sat, Oct 26, 2013 at 8:14 AM, Taavi Burns wrote: > On Mavericks: > $ sudo find / -type f -name \*ffi.h > > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/include/ffi/ffi.h > > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/ffi/ffi.h > /Users/taavi/src/pypy/rpython/translator/c/src/libffi_msvc/ffi.h > /usr/local/Cellar/libffi/3.0.13/lib/libffi-3.0.13/include/ffi.h > > Some quick googling indicates that yes, /usr/include moved into the > platform SDK inside the XCode application. Sounds like one can restore > it by installing the command-line tools (from > > http://stackoverflow.com/questions/19531262/cant-phpize-or-configure-an-extension-in-os-x-10-9-mavericks > ): > xcode-select --install > > Now I have a /usr/include again! > $ ls /usr/include > AssertMacros.h cache.h ? > > I haven't tried building pypy since the upgrade though. > > On Sat, Oct 26, 2013 at 4:25 AM, Maciej Fijalkowski > wrote: > > Hi Kevin, thanks for the feedback! > > > > Do you have maybe an idea how to improve the docs? > > > > about libffi.h - Apple removed /usr/include and moved it somewhere > > else, no clue where. I also don't have access to Mavericks so I can't > > fix it. > > > > On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: > >> Hey everyone, > >> I was trying to compile pypy and made the following mistakes: > >> > >> - On this page: http://pypy.org/download.html I couldn't figure out > whether > >> I was supposed to run all three of these commands: > >> > >> pypy ../../rpython/bin/rpython -Ojit targetpypystandalone # > get > >> the JIT version > >> pypy ../../rpython/bin/rpython -O2 targetpypystandalone # > get > >> the no-jit version > >> pypy ../../rpython/bin/rpython -O2 --sandbox targetpypystandalone # > get > >> the sandbox version > >> > >> or just one. > >> > >> - Once I got it installled I didn't realize there was an additional step > >> after pypy-c was created. > >> > >> - I tried running python package.py -h and that didn't do what I > expected > >> (show help dialog) > >> > >> - I tried running package.py with the right options and it hung with > this > >> message (on both Mac and Centos): > >> > >> sll_os.ll_os_getenv(sPYPY_GENERATIONGC_NURSERY > >> > >> The only Google search results for this are an IRC channel from 2 years > ago. > >> It turned out I didn't want the Sandbox option, I think. > >> > >> - I tried compiling outside the sandbox and got this error: > >> > >> raise NoTTY("Cannot start the debugger when stdout is captured.") > >> > >> It turns out this is a red herring and the real error was way above > that: > >> > >> error: ffi.h: No such file or directory > >> > >> It would be nice if these were more obvious.. of course I expect to run > into > >> issues but the errors could be more clear when they are. > >> > >> Continuing the install process now, will post updates here as I have > them. > >> > >> I am happy to help try and fix these where I can, if you can point me > in the > >> right direction - I have a fair amount of experience working with new > users > >> at Twilio. I have also made exactly one commit to Pypy source code: > >> https://bitbucket.org/pypy/pypy/commits/665baf750859/ Pypy is a lovely > >> project and I'd love to try and help make this process easier. > >> > >> Best, > >> Kevin > >> > >> -- > >> Kevin Burke | Twilio > >> phone: 925.271.7005 | kev.inburke.com > >> > >> _______________________________________________ > >> pypy-dev mailing list > >> pypy-dev at python.org > >> https://mail.python.org/mailman/listinfo/pypy-dev > >> > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev at python.org > > https://mail.python.org/mailman/listinfo/pypy-dev > > > > -- > taa > /*eof*/ > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kev at inburke.com Sat Oct 26 17:27:47 2013 From: kev at inburke.com (Kevin Burke) Date: Sat, 26 Oct 2013 08:27:47 -0700 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Aha, looks like I needed to point C_INCLUDE_PATH to the directory with my custom sqlite.h, all is well on Centos now -- Kevin Burke | Twilio phone: 925.271.7005 | kev.inburke.com On Sat, Oct 26, 2013 at 7:53 AM, Kevin Burke wrote: > Hi Maciej, > I should note - I'm on Mountain Lion so libffi.h is not the issue at the > moment - currently trying to compile this on Centos get the #include > in _sqlite3.py to find my custom sqlite3.h and not the one in > /usr/lib. > > Here is the compile script I am using: > > https://gist.github.com/kevinburke/7170273 > > > -- > Kevin Burke | Twilio > phone: 925.271.7005 | kev.inburke.com > > > On Sat, Oct 26, 2013 at 7:46 AM, Armin Rigo wrote: > >> Hi Kevin, >> >> Thanks for this report ! A quick note though: >> >> On Sat, Oct 26, 2013 at 7:18 AM, Kevin Burke wrote: >> > - Once I got it installled I didn't realize there was an additional step >> > after pypy-c was created. >> >> Note that this step, "package.py", is not required to run the pypy-c >> executable. It's merely a tool that packs all files needed for >> system-wide installation. It also prebuilds a few cffi modules. The >> alternative is to simply use the pypy-c where it is (pypy/goal/pypy-c) >> instead, if you don't need it to be installed system-wide. Symlinks >> pointing to pypy/goal/pypy-c work fine. >> >> >> A bient?t, >> >> Armin. >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rami.chowdhury at gmail.com Sun Oct 27 01:40:16 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Sun, 27 Oct 2013 00:40:16 +0100 Subject: [pypy-dev] error message with windows In-Reply-To: <526C5226.9060101@shaw.ca> References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> Message-ID: On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: > > On 25/10/2013 3:15 PM, Rami Chowdhury wrote: >> >> Hi Trevor, >> >> On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: >>> >>> I get an error message while trying to unzip you program "Please insert >>> the last disk of the multi-disk set?" >>> >>> Where do I find this disk, its not included in the installer package? >> >> Can you please clarify which installer package you downloaded? >> >> > installer pypy-2.1-win32.zip > And which version of Windows are you working on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Sun Oct 27 01:58:06 2013 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Sat, 26 Oct 2013 18:58:06 -0500 Subject: [pypy-dev] Using RPython for a semi-statically-typed language Message-ID: Hello, I am in need of a little assistance. I am thinking of writing a language in RPython. Now, here is what I want: -Polymorphism(C++) without pointers, i.e., Derived can be implicitly cast to Base I am thinking of picking RPython because of the JIT. However, I am worried about the speed. I know the JIT PyPy generates is fast, but if it's the same speed as PyPy itself, someone would easily pick PyPy or Topaz or the like. My question is: For a somewhat statically-typed language like I am planning, are there any extra optimizations I could add to make the JIT work faster? Since the variables have types, I was thinking there might be a way to make it work faster. Thanks in advance! -- Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Sun Oct 27 02:01:14 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sat, 26 Oct 2013 17:01:14 -0700 Subject: [pypy-dev] Using RPython for a semi-statically-typed language In-Reply-To: References: Message-ID: Hi Ryan, I'm not sure this completely answers your question, but the JIT does let you tell it when a class is statically known: https://bitbucket.org/pypy/pypy/src/061674dab6430beb1645f43449fc114a09d58834/rpython/rlib/jit.py?at=default#cl-979so this may help you. In general RPython (and the JIT), are designed for dynamically typed languages, but "semi-statically" may be the same as dynamic for all intents and purposes (I'm not sure). Alex On Sat, Oct 26, 2013 at 4:58 PM, Ryan Gonzalez wrote: > Hello, > > I am in need of a little assistance. I am thinking of writing a language > in RPython. Now, here is what I want: > > -Polymorphism(C++) without pointers, i.e., Derived can be implicitly cast > to Base > > I am thinking of picking RPython because of the JIT. However, I am worried > about the speed. I know the JIT PyPy generates is fast, but if it's the > same speed as PyPy itself, someone would easily pick PyPy or Topaz or the > like. > > My question is: For a somewhat statically-typed language like I am > planning, are there any extra optimizations I could add to make the JIT > work faster? Since the variables have types, I was thinking there might be > a way to make it work faster. > > Thanks in advance! > > -- > Ryan > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From matti.picus at gmail.com Sun Oct 27 05:39:34 2013 From: matti.picus at gmail.com (matti picus) Date: Sun, 27 Oct 2013 07:39:34 +0300 Subject: [pypy-dev] error message with windows In-Reply-To: <526AA86C.1050307@shaw.ca> References: <526AA86C.1050307@shaw.ca> Message-ID: This zip has been downloaded more than 15000 times and you are the first to report this problem. Please try again, and if it fails tell us exactly how you are downloading and opening it, what is your operating system and anything else that many help us to reproduce your specific problem https://bitbucket.org/pypy/pypy/downloads/pypy-2.1-win32.zip Matti On Oct 25, 2013 8:22 PM, "Trevor Langevin" wrote: > > > Hello, > > I'm learning 3D printing, and to use Skeinforge it was recommended to > download your pypy program to use on a windows machine. > > I get an error message while trying to unzip you program "Please insert > the last disk of the multi-disk set?" > > > Where do I find this disk, its not included in the installer package? > > > Trev > > > I could not log on to your bug tracker site, firefox would not let me view > the page as its listed as untrusted. > ______________________________**_________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/**mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sun Oct 27 08:11:05 2013 From: arigo at tunes.org (Armin Rigo) Date: Sun, 27 Oct 2013 09:11:05 +0200 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: Hi Alex, On Sat, Oct 26, 2013 at 5:25 PM, Alex Gaynor wrote: > Yup, if you run `xcode-select --install` everything fixes itself. How about writing down this magic line in the doc? A bient?t, Armin. From rami.chowdhury at gmail.com Sun Oct 27 10:11:22 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Sun, 27 Oct 2013 09:11:22 +0000 Subject: [pypy-dev] error message with windows In-Reply-To: <526CD7FE.6000601@shaw.ca> References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> Message-ID: On Oct 27, 2013 9:08 AM, "Trevor Langevin" wrote: > > It failed in xp, vista, and win 7. > Can you provide any more details? Screenshots? Every step you took and the exact error messages you receive? Also, please reply to the list, not directly to me. > > > On 26/10/2013 4:40 PM, Rami Chowdhury wrote: >> >> >> On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: >> > >> > On 25/10/2013 3:15 PM, Rami Chowdhury wrote: >> >> >> >> Hi Trevor, >> >> >> >> On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: >> >>> >> >>> I get an error message while trying to unzip you program "Please insert >> >>> the last disk of the multi-disk set?" >> >>> >> >>> Where do I find this disk, its not included in the installer package? >> >> >> >> Can you please clarify which installer package you downloaded? >> >> >> >> >> > installer pypy-2.1-win32.zip >> > >> >> And which version of Windows are you working on? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.gaynor at gmail.com Sun Oct 27 15:05:38 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 27 Oct 2013 07:05:38 -0700 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: It's only applicable (as far as I know) when upgrading from 10.8 to 10.9, so our docs don't really seem like the right place for it. Alex On Sun, Oct 27, 2013 at 12:11 AM, Armin Rigo wrote: > Hi Alex, > > On Sat, Oct 26, 2013 at 5:25 PM, Alex Gaynor > wrote: > > Yup, if you run `xcode-select --install` everything fixes itself. > > How about writing down this magic line in the doc? > > > A bient?t, > > Armin. > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Sun Oct 27 22:06:28 2013 From: holger at merlinux.eu (holger krekel) Date: Sun, 27 Oct 2013 21:06:28 +0000 Subject: [pypy-dev] pypi blog shows strange on planet.python.org Message-ID: <20131027210628.GN3973@merlinux.eu> The last "Making coverage.py faster under pypy" blog shows strangely on the Python planet, check it out: http://planet.python.org/ I guess it's not a good idea to include the docutils comments or so. IIRC this didn't happen before. best, holger From rymg19 at gmail.com Sun Oct 27 22:12:51 2013 From: rymg19 at gmail.com (Ryan) Date: Sun, 27 Oct 2013 16:12:51 -0500 Subject: [pypy-dev] Using RPython for a semi-statically-typed language In-Reply-To: References: Message-ID: Thank you!! That will probably help a lot. Even in those special cases, since all types will derive from a common base, it might still show a very slight speed boost. Alex Gaynor wrote: >Hi Ryan, > >I'm not sure this completely answers your question, but the JIT does >let >you tell it when a class is statically known: >https://bitbucket.org/pypy/pypy/src/061674dab6430beb1645f43449fc114a09d58834/rpython/rlib/jit.py?at=default#cl-979so >this may help you. > >In general RPython (and the JIT), are designed for dynamically typed >languages, but "semi-statically" may be the same as dynamic for all >intents >and purposes (I'm not sure). > >Alex > > >On Sat, Oct 26, 2013 at 4:58 PM, Ryan Gonzalez >wrote: > >> Hello, >> >> I am in need of a little assistance. I am thinking of writing a >language >> in RPython. Now, here is what I want: >> >> -Polymorphism(C++) without pointers, i.e., Derived can be implicitly >cast >> to Base >> >> I am thinking of picking RPython because of the JIT. However, I am >worried >> about the speed. I know the JIT PyPy generates is fast, but if it's >the >> same speed as PyPy itself, someone would easily pick PyPy or Topaz or >the >> like. >> >> My question is: For a somewhat statically-typed language like I am >> planning, are there any extra optimizations I could add to make the >JIT >> work faster? Since the variables have types, I was thinking there >might be >> a way to make it work faster. >> >> Thanks in advance! >> >> -- >> Ryan >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> > > >-- >"I disapprove of what you say, but I will defend to the death your >right to >say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >"The people's good is the highest law." -- Cicero >GPG Key fingerprint: 125F 5C67 DFE9 4084 -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rymg19 at gmail.com Sun Oct 27 22:12:52 2013 From: rymg19 at gmail.com (Ryan) Date: Sun, 27 Oct 2013 16:12:52 -0500 Subject: [pypy-dev] Using RPython for a semi-statically-typed language In-Reply-To: References: Message-ID: Thank you!! That will probably help a lot. Even in those special cases, since all types will derive from a common base, it might still show a very slight speed boost. Alex Gaynor wrote: >Hi Ryan, > >I'm not sure this completely answers your question, but the JIT does >let >you tell it when a class is statically known: >https://bitbucket.org/pypy/pypy/src/061674dab6430beb1645f43449fc114a09d58834/rpython/rlib/jit.py?at=default#cl-979so >this may help you. > >In general RPython (and the JIT), are designed for dynamically typed >languages, but "semi-statically" may be the same as dynamic for all >intents >and purposes (I'm not sure). > >Alex > > >On Sat, Oct 26, 2013 at 4:58 PM, Ryan Gonzalez >wrote: > >> Hello, >> >> I am in need of a little assistance. I am thinking of writing a >language >> in RPython. Now, here is what I want: >> >> -Polymorphism(C++) without pointers, i.e., Derived can be implicitly >cast >> to Base >> >> I am thinking of picking RPython because of the JIT. However, I am >worried >> about the speed. I know the JIT PyPy generates is fast, but if it's >the >> same speed as PyPy itself, someone would easily pick PyPy or Topaz or >the >> like. >> >> My question is: For a somewhat statically-typed language like I am >> planning, are there any extra optimizations I could add to make the >JIT >> work faster? Since the variables have types, I was thinking there >might be >> a way to make it work faster. >> >> Thanks in advance! >> >> -- >> Ryan >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev at python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> > > >-- >"I disapprove of what you say, but I will defend to the death your >right to >say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >"The people's good is the highest law." -- Cicero >GPG Key fingerprint: 125F 5C67 DFE9 4084 -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sun Oct 27 22:16:29 2013 From: arigo at tunes.org (Armin Rigo) Date: Sun, 27 Oct 2013 22:16:29 +0100 Subject: [pypy-dev] pypi blog shows strange on planet.python.org In-Reply-To: <20131027210628.GN3973@merlinux.eu> References: <20131027210628.GN3973@merlinux.eu> Message-ID: Hi Holger, On Sun, Oct 27, 2013 at 10:06 PM, holger krekel wrote: > The last "Making coverage.py faster under pypy" blog shows > strangely on the Python planet, check it out: Maybe related: the formatting of http://morepypy.blogspot.com has changed recently, so it may be the case that blogspot tweaked something. (Fwiw http://morepypy.blogspot.com's left pane displays rather badly for me, which is another matter.) A bient?t, Armin. From alex.gaynor at gmail.com Sun Oct 27 22:18:43 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 27 Oct 2013 14:18:43 -0700 Subject: [pypy-dev] pypi blog shows strange on planet.python.org In-Reply-To: References: <20131027210628.GN3973@merlinux.eu> Message-ID: It's not unlikely I screwed up the formatted of the post :) If there's some specific way I can fix it let me know. Alex On Sun, Oct 27, 2013 at 2:16 PM, Armin Rigo wrote: > Hi Holger, > > On Sun, Oct 27, 2013 at 10:06 PM, holger krekel > wrote: > > The last "Making coverage.py faster under pypy" blog shows > > strangely on the Python planet, check it out: > > Maybe related: the formatting of http://morepypy.blogspot.com has > changed recently, so it may be the case that blogspot tweaked > something. (Fwiw http://morepypy.blogspot.com's left pane displays > rather badly for me, which is another matter.) > > > A bient?t, > > Armin. > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Sun Oct 27 22:26:49 2013 From: arigo at tunes.org (Armin Rigo) Date: Sun, 27 Oct 2013 22:26:49 +0100 Subject: [pypy-dev] pypi blog shows strange on planet.python.org In-Reply-To: References: <20131027210628.GN3973@merlinux.eu> Message-ID: Hi Alex, On Sun, Oct 27, 2013 at 10:18 PM, Alex Gaynor wrote: > It's not unlikely I screwed up the formatted of the post :) If there's some > specific way I can fix it let me know. Ah, I see :-) It's because you pasted too much html from the ReST output --- you included the whole style section. Fixed on our blog now. I have no idea if aggregators like planet.python will pick up the update. A bient?t, Armin. From alex.gaynor at gmail.com Sun Oct 27 22:28:30 2013 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Sun, 27 Oct 2013 14:28:30 -0700 Subject: [pypy-dev] pypi blog shows strange on planet.python.org In-Reply-To: References: <20131027210628.GN3973@merlinux.eu> Message-ID: Ahhhh. Thanks for the fix! Alex On Sun, Oct 27, 2013 at 2:26 PM, Armin Rigo wrote: > Hi Alex, > > On Sun, Oct 27, 2013 at 10:18 PM, Alex Gaynor > wrote: > > It's not unlikely I screwed up the formatted of the post :) If there's > some > > specific way I can fix it let me know. > > Ah, I see :-) It's because you pasted too much html from the ReST > output --- you included the whole style section. Fixed on our blog > now. > > I have no idea if aggregators like planet.python will pick up the update. > > > A bient?t, > > Armin. > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084 -------------- next part -------------- An HTML attachment was scrubbed... URL: From rami.chowdhury at gmail.com Mon Oct 28 01:13:25 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Mon, 28 Oct 2013 00:13:25 +0000 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: On Sun, Oct 27, 2013 at 2:05 PM, Alex Gaynor wrote: > It's only applicable (as far as I know) when upgrading from 10.8 to 10.9, so > our docs don't really seem like the right place for it. Having had a look, the PyPy install documentation (pypy/doc/getting-started-python.rst) doesn't have any notes about build-time dependencies on OS X or making sure the developer tools, etc are installed. It's not in the `improve-docs` branch either. AFAIK the only basic requirement to build on OS X is having the Developer Tools, since all the other optional dependencies are installed with them on OS X (I couldn't find a way to get `libbz2.h`, though, any tips?). If that's the case I'll submit a notice to the doc for extra clarity. > > Alex > > > On Sun, Oct 27, 2013 at 12:11 AM, Armin Rigo wrote: >> >> Hi Alex, >> >> On Sat, Oct 26, 2013 at 5:25 PM, Alex Gaynor >> wrote: >> > Yup, if you run `xcode-select --install` everything fixes itself. >> >> How about writing down this magic line in the doc? >> >> >> A bient?t, >> >> Armin. > > > > > -- > "I disapprove of what you say, but I will defend to the death your right to > say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > GPG Key fingerprint: 125F 5C67 DFE9 4084 > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > -- Rami Chowdhury "A mind all logic is like a knife all blade - it makes the hand bleed that uses it." -- Rabindranath Tagore +44-7581-430-517 / +1-408-597-7068 / +88-01771-064063 From rami.chowdhury at gmail.com Mon Oct 28 01:17:29 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Mon, 28 Oct 2013 00:17:29 +0000 Subject: [pypy-dev] error message with windows In-Reply-To: <526CE516.3000807@shaw.ca> References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> <526CE516.3000807@shaw.ca> Message-ID: On Sunday, October 27, 2013 at 10:04, Trevor Langevin wrote: > This email was where I was sent to by your web master. When you simply press "reply" in some email clients then you will send the reply only to the person who responded to you -- in this case me. Please be sure to use "reply all" to include the pypy-dev mailing list. > > Perhaps you should try downloading and installing the package as it failed three times in a row on two different computers, a win 7 computer and win vista. You really haven't given us enough information to help diagnose the problem. What is the exact error message? Which exact version of Windows is it? Most of the PyPy developers don't use Windows as their main platform so it's not so easy to track down errors, and we would appreciate all the help you can give us. > > On 27/10/2013 2:11 AM, Rami Chowdhury wrote: > > > > On Oct 27, 2013 9:08 AM, "Trevor Langevin" wrote: > > > > > > It failed in xp, vista, and win 7. > > Can you provide any more details? Screenshots? Every step you took and the exact error messages you receive? > > Also, please reply to the list, not directly to me. > > > > > > > > > On 26/10/2013 4:40 PM, Rami Chowdhury wrote: > > > > > > > > > > > > On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: > > > > > > > > > > On 25/10/2013 3:15 PM, Rami Chowdhury wrote: > > > > > > > > > > > > Hi Trevor, > > > > > > > > > > > > On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: > > > > > > > > > > > > > > I get an error message while trying to unzip you program "Please insert > > > > > > > the last disk of the multi-disk set?" > > > > > > > > > > > > > > Where do I find this disk, its not included in the installer package? > > > > > > > > > > > > Can you please clarify which installer package you downloaded? > > > > > installer pypy-2.1-win32.zip > > > > > > > > > > > > And which version of Windows are you working on? From cco at dydax.com Mon Oct 28 01:34:41 2013 From: cco at dydax.com (Chris Olds) Date: Sun, 27 Oct 2013 17:34:41 -0700 Subject: [pypy-dev] error message with windows In-Reply-To: References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> <526CE516.3000807@shaw.ca> Message-ID: <857F99E7-3CD9-4C96-9DD9-B12106B86737@dydax.com> This is the message you get when the zipfile is truncated. The directory at the end of the file isn't found, which used to happen with multi-disk archives, so you're being prompted to insert it. This used to be reasonable, but now is just confusing. /cco On Oct 27, 2013, at 5:17 PM, Rami Chowdhury wrote: > On Sunday, October 27, 2013 at 10:04, Trevor Langevin wrote: >> This email was where I was sent to by your web master. > > When you simply press "reply" in some email clients then you will send the reply only to the person who responded to you -- in this case me. Please be sure to use "reply all" to include the pypy-dev mailing list. >> >> Perhaps you should try downloading and installing the package as it failed three times in a row on two different computers, a win 7 computer and win vista. > > You really haven't given us enough information to help diagnose the problem. What is the exact error message? Which exact version of Windows is it? Most of the PyPy developers don't use Windows as their main platform so it's not so easy to track down errors, and we would appreciate all the help you can give us. >> >> On 27/10/2013 2:11 AM, Rami Chowdhury wrote: >>> >>> On Oct 27, 2013 9:08 AM, "Trevor Langevin" wrote: >>>> >>>> It failed in xp, vista, and win 7. >>> Can you provide any more details? Screenshots? Every step you took and the exact error messages you receive? >>> Also, please reply to the list, not directly to me. >>>> >>>> >>>> On 26/10/2013 4:40 PM, Rami Chowdhury wrote: >>>>> >>>>> >>>>> On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: >>>>>> >>>>>> On 25/10/2013 3:15 PM, Rami Chowdhury wrote: >>>>>>> >>>>>>> Hi Trevor, >>>>>>> >>>>>>> On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: >>>>>>>> >>>>>>>> I get an error message while trying to unzip you program "Please insert >>>>>>>> the last disk of the multi-disk set?" >>>>>>>> >>>>>>>> Where do I find this disk, its not included in the installer package? >>>>>>> >>>>>>> Can you please clarify which installer package you downloaded? >>>>>> installer pypy-2.1-win32.zip >>>>> >>>>> >>>>> And which version of Windows are you working on? > > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev From rami.chowdhury at gmail.com Mon Oct 28 01:54:24 2013 From: rami.chowdhury at gmail.com (Rami Chowdhury) Date: Mon, 28 Oct 2013 00:54:24 +0000 Subject: [pypy-dev] error message with windows In-Reply-To: <857F99E7-3CD9-4C96-9DD9-B12106B86737@dydax.com> References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> <526CE516.3000807@shaw.ca> <857F99E7-3CD9-4C96-9DD9-B12106B86737@dydax.com> Message-ID: On Oct 28, 2013 12:34 AM, "Chris Olds" wrote: > > This is the message you get when the zipfile is truncated. So most likely Trevor's download was incomplete or corrupted? > The directory at the end of the file isn't found, which used to happen with multi-disk archives, so you're being prompted to insert it. > This used to be reasonable, but now is just confusing. > > /cco > > On Oct 27, 2013, at 5:17 PM, Rami Chowdhury wrote: > > > On Sunday, October 27, 2013 at 10:04, Trevor Langevin wrote: > >> This email was where I was sent to by your web master. > > > > When you simply press "reply" in some email clients then you will send the reply only to the person who responded to you -- in this case me. Please be sure to use "reply all" to include the pypy-dev mailing list. > >> > >> Perhaps you should try downloading and installing the package as it failed three times in a row on two different computers, a win 7 computer and win vista. > > > > You really haven't given us enough information to help diagnose the problem. What is the exact error message? Which exact version of Windows is it? Most of the PyPy developers don't use Windows as their main platform so it's not so easy to track down errors, and we would appreciate all the help you can give us. > >> > >> On 27/10/2013 2:11 AM, Rami Chowdhury wrote: > >>> > >>> On Oct 27, 2013 9:08 AM, "Trevor Langevin" wrote: > >>>> > >>>> It failed in xp, vista, and win 7. > >>> Can you provide any more details? Screenshots? Every step you took and the exact error messages you receive? > >>> Also, please reply to the list, not directly to me. > >>>> > >>>> > >>>> On 26/10/2013 4:40 PM, Rami Chowdhury wrote: > >>>>> > >>>>> > >>>>> On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: > >>>>>> > >>>>>> On 25/10/2013 3:15 PM, Rami Chowdhury wrote: > >>>>>>> > >>>>>>> Hi Trevor, > >>>>>>> > >>>>>>> On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: > >>>>>>>> > >>>>>>>> I get an error message while trying to unzip you program "Please insert > >>>>>>>> the last disk of the multi-disk set?" > >>>>>>>> > >>>>>>>> Where do I find this disk, its not included in the installer package? > >>>>>>> > >>>>>>> Can you please clarify which installer package you downloaded? > >>>>>> installer pypy-2.1-win32.zip > >>>>> > >>>>> > >>>>> And which version of Windows are you working on? > > > > > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev at python.org > > https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From cco at dydax.com Mon Oct 28 04:04:44 2013 From: cco at dydax.com (Chris Olds) Date: Sun, 27 Oct 2013 20:04:44 -0700 Subject: [pypy-dev] error message with windows In-Reply-To: References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> <526CE516.3000807@shaw.ca> <857F99E7-3CD9-4C96-9DD9-B12106B86737@dydax.com> Message-ID: <4F7ABB23-254A-4732-A174-2A056AB1FA3F@dydax.com> Yes, that is my assumption. /cco On Oct 27, 2013, at 5:54 PM, Rami Chowdhury wrote: > > On Oct 28, 2013 12:34 AM, "Chris Olds" wrote: > > > > This is the message you get when the zipfile is truncated. > > So most likely Trevor's download was incomplete or corrupted? > > > The directory at the end of the file isn't found, which used to happen with multi-disk archives, so you're being prompted to insert it. > > This used to be reasonable, but now is just confusing. > > > > /cco > > > > On Oct 27, 2013, at 5:17 PM, Rami Chowdhury wrote: > > > > > On Sunday, October 27, 2013 at 10:04, Trevor Langevin wrote: > > >> This email was where I was sent to by your web master. > > > > > > When you simply press "reply" in some email clients then you will send the reply only to the person who responded to you -- in this case me. Please be sure to use "reply all" to include the pypy-dev mailing list. > > >> > > >> Perhaps you should try downloading and installing the package as it failed three times in a row on two different computers, a win 7 computer and win vista. > > > > > > You really haven't given us enough information to help diagnose the problem. What is the exact error message? Which exact version of Windows is it? Most of the PyPy developers don't use Windows as their main platform so it's not so easy to track down errors, and we would appreciate all the help you can give us. > > >> > > >> On 27/10/2013 2:11 AM, Rami Chowdhury wrote: > > >>> > > >>> On Oct 27, 2013 9:08 AM, "Trevor Langevin" wrote: > > >>>> > > >>>> It failed in xp, vista, and win 7. > > >>> Can you provide any more details? Screenshots? Every step you took and the exact error messages you receive? > > >>> Also, please reply to the list, not directly to me. > > >>>> > > >>>> > > >>>> On 26/10/2013 4:40 PM, Rami Chowdhury wrote: > > >>>>> > > >>>>> > > >>>>> On Oct 27, 2013 12:37 AM, "Trevor Langevin" wrote: > > >>>>>> > > >>>>>> On 25/10/2013 3:15 PM, Rami Chowdhury wrote: > > >>>>>>> > > >>>>>>> Hi Trevor, > > >>>>>>> > > >>>>>>> On Friday, October 25, 2013 at 18:20, Trevor Langevin wrote: > > >>>>>>>> > > >>>>>>>> I get an error message while trying to unzip you program "Please insert > > >>>>>>>> the last disk of the multi-disk set?" > > >>>>>>>> > > >>>>>>>> Where do I find this disk, its not included in the installer package? > > >>>>>>> > > >>>>>>> Can you please clarify which installer package you downloaded? > > >>>>>> installer pypy-2.1-win32.zip > > >>>>> > > >>>>> > > >>>>> And which version of Windows are you working on? > > > > > > > > > _______________________________________________ > > > pypy-dev mailing list > > > pypy-dev at python.org > > > https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From tlangevin at shaw.ca Mon Oct 28 07:57:46 2013 From: tlangevin at shaw.ca (Trevor Langevin) Date: Sun, 27 Oct 2013 23:57:46 -0700 Subject: [pypy-dev] error message with windows In-Reply-To: <4F7ABB23-254A-4732-A174-2A056AB1FA3F@dydax.com> References: <526AA86C.1050307@shaw.ca> <3EE6B7116D8748ED9352DA34A6652652@gmail.com> <526C5226.9060101@shaw.ca> <526CD7FE.6000601@shaw.ca> <526CE516.3000807@shaw.ca> <857F99E7-3CD9-4C96-9DD9-B12106B86737@dydax.com> <4F7ABB23-254A-4732-A174-2A056AB1FA3F@dydax.com> Message-ID: <526E0AEA.30300@shaw.ca> An HTML attachment was scrubbed... URL: From wlavrijsen at lbl.gov Tue Oct 29 00:58:51 2013 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Mon, 28 Oct 2013 16:58:51 -0700 (PDT) Subject: [pypy-dev] trying out STM for some numbers on more cores Message-ID: Hi, last week I inherited a dual 8 core E5-2650 box, so 16 physical cores. For fun I wanted to redo the STM numbers from the blog posting on more cores. I killed the printout coming from rpython, as besides giving unnecessary syncing, it also causes crashes beyond 12 threads or so. Here are my numbers: Reference CPython, 1 thread: 271.17 Reference PyPy-2.1, 1 thread: 9.63 (Note above for whatever reason my CPython run is slower that in the blog post, even as the pypy run is a lot faster?) Freshly-translated (1385fb758727+ (stmgc-c4)): richards.py, 100 iterations each, printout from rpython removed: Threads avg. time speedup 1 358.48 1.0 2 244.86 1.5 4 162.87 2.2 8 141.48 2.5 16 127.43 2.8 32 146.57 2.4 Note that the speedup numbers above are less than what was posted on the blog, and scaling with hyperthreads isn't working well. Has any attempt been made to pin threads? Going beyond 32 crashes in: #1 0x0000000000d29026 in pypy_g_start_new_thread () #2 0x0000000000498347 in pypy_g_BuiltinActivation_UwS_ObjSpace_W_Root_W_Root_W_R () #3 0x000000000085a8c1 in pypy_g_BuiltinCode_funcrun_obj () #4 0x0000000000858255 in pypy_g_funccall_valuestack__AccessDirect_None () #5 0x0000000000d73d86 in pypy_g_CALL_METHOD__AccessDirect_star_1 () #6 0x000000000089d3a3 in pypy_g_dispatch_bytecode__AccessDirect_None () #7 0x00000000008a34b5 in pypy_g_handle_bytecode__AccessDirect_None () #8 0x0000000000cfe311 in pypy_g_dispatch__AccessDirect_None_stm () #9 0x000000000136f974 in pypy_g__stm_callback_4 () #10 0x00000000014ea1cf in stm_perform_transaction () #11 0x000000000050e62e in pypy_g_ll_invoke_stm__pypy_objspace_std_frame_StdObjSpa () #12 0x00000000011ec935 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter () #13 0x0000000000874bda in pypy_g_PyFrame_run () Still trying to see whether I can get PyPy to run on the MIC. :) Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From arigo at tunes.org Tue Oct 29 09:38:39 2013 From: arigo at tunes.org (Armin Rigo) Date: Tue, 29 Oct 2013 09:38:39 +0100 Subject: [pypy-dev] Fwd: LLVM stackmaps In-Reply-To: References: Message-ID: Hi all, hi mjacob, Someone interested in the port to llvm could have a look at this: http://llvm-reviews.chandlerc.com/file/data/qnievreabgj3qqnco3ax/PHID-FILE-ofdo5nanmf23mhblqcga/D1981.diff It adds two different things. The first is stackmaps, which could replace shadowstack/asmgcc for regular translation. The second is patchpoints, which together with stackmaps could potentially be used for a llvm JIT backend's guards. A bient?t, Armin. From tismer at stackless.com Tue Oct 29 15:02:08 2013 From: tismer at stackless.com (Christian Tismer) Date: Tue, 29 Oct 2013 15:02:08 +0100 Subject: [pypy-dev] A few notes trying to compile pypy In-Reply-To: References: Message-ID: <526FBFE0.4010208@stackless.com> On 26/10/13 16:35, Armin Rigo wrote: > Hi, > > On Sat, Oct 26, 2013 at 10:25 AM, Maciej Fijalkowski wrote: >> about libffi.h - Apple removed /usr/include and moved it somewhere >> else, no clue where. I also don't have access to Mavericks so I can't >> fix it. > Ooooh. Traditionally, "libffi.h" was put in unexpectedly varying > directories on every Linux distribution, just to make sure there was > no reliable way to find it, but that was resolved some time ago with > pkg-config. Apple's move is on a completely different level: killing > the whole /usr/include is a perfect way to break compilation for every > open source project under the sun. I'm impressed. > > That is much less of a problem because there is homebrew, and I use it for almost everything that I don't want to build/take care of. About mavericks: can I help you out? Works fine in a virtual machine using parallels, if you don't want to change the system. cheers - chris -- Christian Tismer :^) Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tismer at stackless.com Tue Oct 29 14:56:17 2013 From: tismer at stackless.com (Christian Tismer) Date: Tue, 29 Oct 2013 14:56:17 +0100 Subject: [pypy-dev] progress with numpy and removal of numpy.py In-Reply-To: <525FA70B.6010009@gmail.com> References: <525CC15F.70103@gmail.com> <525FA70B.6010009@gmail.com> Message-ID: <526FBE81.6050306@stackless.com> On 17/10/13 10:59, Antonio Cuni wrote: > Hi, > > On 15/10/13 12:45, matti picus wrote: >> and what about cloning the numpy repo into bitbucket/pypy to make it >> more of a >> "pypy owned" thing? > > I think it's a good idea. > > I propose the following: > > 1) we move your repo to bitbucket/pypy/numpypy > > 2) we package numpypy, so that people can just do "pip install numpypy" > > 3) once numpypy is installed, we no longer require the ugly "import > numpypy"; a simple "import numpy" will just work. > > 4) for some time at least, we distribute a numpypy.py so that when > imported it prints an error message which explain how to get the newer > numpypy Just a note: A repo from github is definately a git repo, and it stays in git format when moved to bitbucket. May not be relevant, I just wanted to mention that. cheers - Chris p.s.: After having worked with both for quite a long time, I now would prefer if pypy and python would use git ;-) -- Christian Tismer :^) Software Consulting : Have a break! Take a ride on Python's Karl-Liebknecht-Str. 121 : *Starship* http://starship.python.net/ 14482 Potsdam : PGP key -> http://pgp.uni-mainz.de phone +49 173 24 18 776 fax +49 (30) 700143-0023 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From ddvento at ucar.edu Tue Oct 29 18:21:10 2013 From: ddvento at ucar.edu (Davide Del Vento) Date: Tue, 29 Oct 2013 11:21:10 -0600 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: Message-ID: <526FEE86.8000408@ucar.edu> On 10/28/2013 05:58 PM, wlavrijsen at lbl.gov wrote: > Has any attempt > been made to pin threads? I don't know. But I do know that processor/thread binding (if that is what you mean by "pin") is *extremely* important in Sandy Bridge, even more than on previous archs. And, oddly enough, in my experience is more difficult to get it right than on previous arch. Unfortunately, the runtime I have most experience with is very different from the standard gcc-pthreads-fork which I believe would be relevant here, so my experience is useless in this context. Good luck finding the way to do this right, keep us posted! Dav From arigo at tunes.org Tue Oct 29 22:54:53 2013 From: arigo at tunes.org (Armin Rigo) Date: Tue, 29 Oct 2013 22:54:53 +0100 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: <526FEE86.8000408@ucar.edu> References: <526FEE86.8000408@ucar.edu> Message-ID: Hi Davide, On Tue, Oct 29, 2013 at 6:21 PM, Davide Del Vento wrote: > I don't know. But I do know that processor/thread binding (if that is what > you mean by "pin") is *extremely* important in Sandy Bridge, even more than > on previous archs. And, oddly enough, in my experience is more difficult to > get it right than on previous arch. Can you point to more information about this? A bient?t, Armin. From wlavrijsen at lbl.gov Wed Oct 30 01:22:16 2013 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Tue, 29 Oct 2013 17:22:16 -0700 (PDT) Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: <526FEE86.8000408@ucar.edu> References: <526FEE86.8000408@ucar.edu> Message-ID: Davide, > I don't know. But I do know that processor/thread binding (if that is what > you mean by "pin") is what I meant. :) But a q&d implementation does not seem to make much difference other than for 8 and 16 threads, where it helps a bit. Running some more, I noticed that there are plenty of other overheads and the 'avg. time' doesn't get anywhere near stable until the number of iterations is in the 1000s (I used 100 before). iterations 16 threads 32 threads PyPy-2.1 100 127.43 146.57 9.63 200 77.59 86.37 7.80 500 46.92 49.12 6.82 1000 36.51 33.80 6.29 2000 32.18 28.69 6.40 The numbers are closer together, and HT now helps (note that the "slowdown" for 2000 iterations for 2.1 is not significant; I should run this multiple times and average, but this is just for fun). It is obvious, though, that overheads are larger for STM atm, and are therefore important for longer. The differences at larger number of iterations are much less for smaller numbers of threads (and zero for 1 thread). Intuitively that makes sense. It also says that 16 threads can give a 11x speedup if there's enough work to do. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From matti.picus at gmail.com Wed Oct 30 06:53:15 2013 From: matti.picus at gmail.com (Matti Picus) Date: Wed, 30 Oct 2013 07:53:15 +0200 Subject: [pypy-dev] numpy repo under pypy on bitbucket Message-ID: <52709ECB.7090600@gmail.com> Thanks to Brian Kearns, the pypy-compat branch of the numpy repo at https://bitbucket.org/pypy/numpy is starting to become the prefrered method of using numpy on pypy. Note this is a git repo. All commiters to pypy should have a commit bit here, let me know if something needs to be changed. Also, I enabled a commit hook so commits should appear in IRC, if this gets excessive it can be turned off. Waiting for your contributions, Matti From ddvento at ucar.edu Wed Oct 30 18:59:43 2013 From: ddvento at ucar.edu (Davide Del Vento) Date: Wed, 30 Oct 2013 11:59:43 -0600 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> Message-ID: <5271490F.7050109@ucar.edu> Hi Armin, On 10/29/2013 03:54 PM, Armin Rigo wrote: > On Tue, Oct 29, 2013 at 6:21 PM, Davide Del Vento wrote: >> I don't know. But I do know that processor/thread binding (if that is what >> you mean by "pin") is *extremely* important in Sandy Bridge, even more than >> on previous archs. And, oddly enough, in my experience is more difficult to >> get it right than on previous arch. > > Can you point to more information about this? Well, as I said "in my experience". I can talk about it right here and some runs I did, if you so desire, but I can't point to anybody saying so "officially". I can point you to the following, which also shows that the runtime we use is pretty different (MPI and OpenMP, often from Intel or other proprietary compilers): https://www2.cisl.ucar.edu/resources/yellowstone/using_resources/runningjobs#env_variables Ciao, Davide From ddvento at ucar.edu Wed Oct 30 19:06:00 2013 From: ddvento at ucar.edu (Davide Del Vento) Date: Wed, 30 Oct 2013 12:06:00 -0600 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> Message-ID: <52714A88.8050708@ucar.edu> Hi Wim, Thanks for posting your numbers. I think they are interesting and the 11x speedup for 16 threads is not bad, however the overhead of STM is still too high compared to PyPy. Maybe you need also a larger dataset, besides a longer time? > I should run this multiple > times and average, but this is just for fun I think for this purpose you need the best timing, not the average, especially if you are using a desktop/laptop. The best timing is something that happened and therefore "can happen". The average is affected by a variety of other things which may be running on your machine, which are better left out for this purpose (it's interesting to study them in order to see if one can eliminate them on a production environment, but that's a completely different job). Cheers, Davide From amauryfa at gmail.com Wed Oct 30 19:45:01 2013 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 30 Oct 2013 19:45:01 +0100 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: <52714A88.8050708@ucar.edu> References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: 2013/10/30 Davide Del Vento > Hi Wim, > > Thanks for posting your numbers. I think they are interesting and the 11x > speedup for 16 threads is not bad, however the overhead of STM is still too > high compared to PyPy. Maybe you need also a larger dataset, besides a > longer time? > > > > I should run this multiple > > times and average, but this is just for fun > > I think for this purpose you need the best timing, not the average, > especially if you are using a desktop/laptop. The best timing is something > that happened and therefore "can happen". The average is affected by a > variety of other things which may be running on your machine, which are > better left out for this purpose (it's interesting to study them in order > to see if one can eliminate them on a production environment, but that's a > completely different job). > Also, the process should perform 1000 iterations before you start the timings. The JIT needs a lot of iterations to be warm-up correctly. -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From taavi.burns at gmail.com Wed Oct 30 20:37:12 2013 From: taavi.burns at gmail.com (Taavi Burns) Date: Wed, 30 Oct 2013 15:37:12 -0400 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: Unless I missed something (possible!) the JIT and STM are mutually exclusive (until implemented). -- taa /*eof*/ > On Oct 30, 2013, at 14:45, "Amaury Forgeot d'Arc" wrote: > > 2013/10/30 Davide Del Vento >> Hi Wim, >> >> Thanks for posting your numbers. I think they are interesting and the 11x speedup for 16 threads is not bad, however the overhead of STM is still too high compared to PyPy. Maybe you need also a larger dataset, besides a longer time? >> >> >> > I should run this multiple >> > times and average, but this is just for fun >> >> I think for this purpose you need the best timing, not the average, especially if you are using a desktop/laptop. The best timing is something that happened and therefore "can happen". The average is affected by a variety of other things which may be running on your machine, which are better left out for this purpose (it's interesting to study them in order to see if one can eliminate them on a production environment, but that's a completely different job). > > Also, the process should perform 1000 iterations before you start the timings. > The JIT needs a lot of iterations to be warm-up correctly. > > -- > Amaury Forgeot d'Arc > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Wed Oct 30 21:20:16 2013 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Wed, 30 Oct 2013 21:20:16 +0100 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: 2013/10/30 Taavi Burns > Unless I missed something (possible!) the JIT and STM are mutually > exclusive (until implemented). > In the last post about STM: http://morepypy.blogspot.ch/2013/10/update-on-stm.html "For comparison, disabling the JIT gives 492ms on PyPy-2.1 and 538ms on PyPy-STM." JIT on PyPy-STM gives only a slight improvement so far, but it will definitely slow down things a lot during the tracing phase. -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From wlavrijsen at lbl.gov Wed Oct 30 23:58:16 2013 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 30 Oct 2013 15:58:16 -0700 (PDT) Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: Amaury, On Wed, 30 Oct 2013, Amaury Forgeot d'Arc wrote: > Also, the process should perform 1000 iterations before you start the > timings. > The JIT needs a lot of iterations to be warm-up correctly. so, each 'iteration' that I had in the table contains an inner loop that is itself JIT-ed (not verified, but just compare the PyPy-2.1 v.s. CPython numbers to see how well that works; likewise, if I switch off the JIT, the result is an xx slowdown compared to CPython). Thus, if '100 iterations,' the hurt should only be in the first iteration. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From wlavrijsen at lbl.gov Thu Oct 31 00:14:49 2013 From: wlavrijsen at lbl.gov (wlavrijsen at lbl.gov) Date: Wed, 30 Oct 2013 16:14:49 -0700 (PDT) Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: <52714A88.8050708@ucar.edu> References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: Davide, > Thanks for posting your numbers. I think they are interesting and the 11x > speedup for 16 threads is not bad, however the overhead of STM is still too > high compared to PyPy. well, yes and no: richards.py runs 30x faster on PyPy than on CPython. The more typical speedup of PyPy is 5x, so If I can get an 11x speedup instead, I'm already pretty happy. In particular, my main interest is as always legacy C++. Remi's thesis shows how one can build higher level structs to control commits. Iow., I can offer the Python user thread-safe access to non-thread-safe C++ without forcing a use pattern on him. Now, if the bulk of the time spent is not in Python in the first place, the overhead may very well not be "too high." Needs to be proven, and of course the lower the overhead the better, but I'm rather optimistic. :) > I think for this purpose you need the best timing, not the average, > especially if you are using a desktop/laptop. Yes, I know: with Intel's version of OpenMP for example, affinity is set to threads on start, not on creation. Short benchmarks tend to be consistently off in individual runs when the assignment ends up unbalanced. The point was more that the last couple of digits in the timing, although printed, are largely noise. And thus that PyPy-2.1 didn't start slowing down for 2000 iterations as the numbers otherwise suggest. Best regards, Wim -- WLavrijsen at lbl.gov -- +1 (510) 486 6411 -- www.lavrijsen.net From taavi.burns at gmail.com Wed Oct 30 23:52:44 2013 From: taavi.burns at gmail.com (Taavi Burns) Date: Wed, 30 Oct 2013 18:52:44 -0400 Subject: [pypy-dev] trying out STM for some numbers on more cores In-Reply-To: References: <526FEE86.8000408@ucar.edu> <52714A88.8050708@ucar.edu> Message-ID: Best news (new to me anyway!) I've read all day. :) Thanks! On Wed, Oct 30, 2013 at 4:20 PM, Amaury Forgeot d'Arc wrote: > 2013/10/30 Taavi Burns >> >> Unless I missed something (possible!) the JIT and STM are mutually >> exclusive (until implemented). > > > In the last post about STM: > http://morepypy.blogspot.ch/2013/10/update-on-stm.html > "For comparison, disabling the JIT gives 492ms on PyPy-2.1 and 538ms on > PyPy-STM." > > JIT on PyPy-STM gives only a slight improvement so far, but it will > definitely slow down things a lot during the tracing phase. > > > -- > Amaury Forgeot d'Arc -- taa /*eof*/ From amauryfa at gmail.com Thu Oct 31 00:40:39 2013 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 31 Oct 2013 00:40:39 +0100 Subject: [pypy-dev] numpy repo under pypy on bitbucket In-Reply-To: <52709ECB.7090600@gmail.com> References: <52709ECB.7090600@gmail.com> Message-ID: Hi, 2013/10/30 Matti Picus > Thanks to Brian Kearns, the pypy-compat branch of the numpy repo at > https://bitbucket.org/pypy/**numpy < > https://bitbucket.org/pypy/**numpy/ > > is starting to become the prefrered method of using numpy on pypy. Note > this is a git repo. > Will nightly builds and pypy release still contain a full numpy module? Or is it necessary to install it manually? -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail2shine at qq.com Thu Oct 31 09:32:22 2013 From: mail2shine at qq.com (=?ISO-8859-1?B?S2FTaGluaW5n?=) Date: Thu, 31 Oct 2013 16:32:22 +0800 Subject: [pypy-dev] pip install rbtree fail in pypy env! Message-ID: pypy 2.1 pip install rbtree >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Downloading/unpacking rbtree Running setup.py egg_info for package rbtree Installing collected packages: rbtree Running setup.py install for rbtree building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 Complete output from command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7: running install running build running build_ext building 'rbtree' extension cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree_impl.c -o build/temp.linux-x86_64-2.7/src/rbtree_impl.o cc -O2 -fPIC -Wimplicit -I./src -I/pypy/include -c src/rbtree.c -o build/temp.linux-x86_64-2.7/src/rbtree.o src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': src/rbtree.c:2427:48: error: 'PyRange_Type' undeclared (first use in this function) src/rbtree.c:2427:48: note: each undeclared identifier is reported only once for each function it appears in src/rbtree.c: In function '__Pyx_GetException': src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4993:5: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4994:5: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4995:5: error: 'PyThreadState' has no member named 'exc_traceback' src/rbtree.c:4996:11: error: 'PyThreadState' has no member named 'exc_type' src/rbtree.c:4997:11: error: 'PyThreadState' has no member named 'exc_value' src/rbtree.c:4998:11: error: 'PyThreadState' has no member named 'exc_traceback' error: command 'cc' failed with exit status 1 ---------------------------------------- Command /pypy/bin/pypy -c "import setuptools;__file__='/pypy/build/rbtree/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-s2EnAa-record/install-record.txt --install-headers /pypy/include/site/python2.7 failed with error code 1 in /pypy/build/rbtree Storing complete log in /home/chenjiaming/.pip/pip.log -------------- next part -------------- An HTML attachment was scrubbed... URL: From arigo at tunes.org Thu Oct 31 09:53:43 2013 From: arigo at tunes.org (Armin Rigo) Date: Thu, 31 Oct 2013 09:53:43 +0100 Subject: [pypy-dev] pip install rbtree fail in pypy env! In-Reply-To: References: Message-ID: Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining wrote: > src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bient?t, Armin. From mail2shine at qq.com Thu Oct 31 10:11:09 2013 From: mail2shine at qq.com (=?utf-8?B?S2FTaGluaW5n?=) Date: Thu, 31 Oct 2013 17:11:09 +0800 Subject: [pypy-dev] pip install rbtree fail in pypy env! In-Reply-To: References: Message-ID: Tks! but >>>>>>>>>>>>>>> This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). >>>>>>>>>>>>>>> why? ------------------ Original ------------------ From: "Armin Rigo";; Date: Thu, Oct 31, 2013 04:53 PM To: "KaShining"; Cc: "pypy-dev"; Subject: Re: [pypy-dev] pip install rbtree fail in pypy env! Hi KaShining, On Thu, Oct 31, 2013 at 9:32 AM, KaShining wrote: > src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': This is using Cython. Compiling Cython modules with PyPy kind-of-works but is shaky. You may have more luck if you upgrade Cython to the latest version, but I don't know the details myself. Note that according to the name "rbtree", it's about a data structure that --- I guess --- is meant to contain tons of references to Python objects. This is a typical example of a program that will be incredibly slower on PyPy if used as a C extension module (via cpyext). You'd get much better results by running a pure Python version of it. Check "rbtree" to see if it already has a pure Python version. A bient?t, Armin. -------------- next part -------------- An HTML attachment was scrubbed... URL: From amauryfa at gmail.com Thu Oct 31 11:22:54 2013 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 31 Oct 2013 11:22:54 +0100 Subject: [pypy-dev] pip install rbtree fail in pypy env! In-Reply-To: References: Message-ID: 2013/10/31 KaShining > Tks! > > but > > >>>>>>>>>>>>>>> > This is a typical example of a program that will be > incredibly slower on PyPy if used as a C extension module (via > cpyext). > >>>>>>>>>>>>>>> > > why? > See this blog post: http://morepypy.blogspot.ch/2011/05/numpy-follow-up.html specially the paragraph starting with "cpyext is slow" > > > ------------------ Original ------------------ > *From: * "Armin Rigo";; > *Date: * Thu, Oct 31, 2013 04:53 PM > *To: * "KaShining"; ** > *Cc: * "pypy-dev"; ** > *Subject: * Re: [pypy-dev] pip install rbtree fail in pypy env! > > Hi KaShining, > > On Thu, Oct 31, 2013 at 9:32 AM, KaShining wrote: > > src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': > > This is using Cython. Compiling Cython modules with PyPy > kind-of-works but is shaky. You may have more luck if you upgrade > Cython to the latest version, but I don't know the details myself. > > Note that according to the name "rbtree", it's about a data structure > that --- I guess --- is meant to contain tons of references to Python > objects. This is a typical example of a program that will be > incredibly slower on PyPy if used as a C extension module (via > cpyext). You'd get much better results by running a pure Python > version of it. Check "rbtree" to see if it already has a pure Python > version. > > > A bient?t, > > Armin. > > _______________________________________________ > pypy-dev mailing list > pypy-dev at python.org > https://mail.python.org/mailman/listinfo/pypy-dev > > -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From davewats at cisco.com Thu Oct 31 17:10:32 2013 From: davewats at cisco.com (Dave Watson (davewats)) Date: Thu, 31 Oct 2013 16:10:32 +0000 Subject: [pypy-dev] Question on installing pypy and the source code .. Message-ID: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com> Hi Developers .. I have been looking at trying to install the pypy interpreter from source code as suggested from the: http://pypy.org website for the very first time. We don't have any previous version of the pypy interpreter installed on our systems at Cisco .. I ran into a bit of an issue in trying to follow the instructions. As was suggested I tried to use the "building from Source" code instructions as I wanted to build it from scratch as we have a somewhat unique environment, so it is usually better to try and build the tools from the source code .. All the build instructions seem to assume that you have a previous version of pypy already installed, in my case, I don't have a previous version of pypy already installed and am trying to create the first one.. :( .. I there a set of instructions for very first time builders ?? Thanks for any help that you can offer .. Cheers Dave W David Watson STEP Tools Master Cisco System 2000 Inovation Drive Ottawa, On K2W 3E8 PH: 613 254-3377 [cid:image001.jpg at 01CDC1A7.C1732360] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 17897 bytes Desc: image001.jpg URL: From wizzat at gmail.com Thu Oct 31 17:25:46 2013 From: wizzat at gmail.com (Mark Roberts) Date: Thu, 31 Oct 2013 09:25:46 -0700 Subject: [pypy-dev] Question on installing pypy and the source code In-Reply-To: References: Message-ID: <2A827429-A93E-407E-9514-6773091189BB@gmail.com> I've had really good luck installing pypy and development environments with pyenv. It includes recipes for installing pretty much any version of Python you might need. -Mark > Hi Developers .. > > I have been looking at trying to install the pypy interpreter from source code as suggested from the: http://pypy.org website for the very first time. We don't have any previous version of the pypy interpreter installed on our systems at Cisco .. > > I ran into a bit of an issue in trying to follow the instructions. As was suggested I tried to use the "building from Source" code instructions as I wanted to build it from scratch as we have a somewhat unique environment, so it is usually better to try and build the tools from the source code .. > > All the build instructions seem to assume that you have a previous version of pypy already installed, in my case, I don't have a previous version of pypy already installed and am trying to create the first one.. :( .. > > I there a set of instructions for very first time builders ?? > > Thanks for any help that you can offer .. > > Cheers > > Dave W > > > David Watson > STEP Tools Master > Cisco System > 2000 Inovation Drive > Ottawa, On K2W 3E8 > PH: 613 254-3377 > [cid:image001.jpg at 01CDC1A7.C1732360] From amauryfa at gmail.com Thu Oct 31 17:49:38 2013 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 31 Oct 2013 17:49:38 +0100 Subject: [pypy-dev] Question on installing pypy and the source code In-Reply-To: <2A827429-A93E-407E-9514-6773091189BB@gmail.com> References: <2A827429-A93E-407E-9514-6773091189BB@gmail.com> Message-ID: 2013/10/31 Mark Roberts > > All the build instructions seem to assume that you have a previous > version of pypy already installed, in my case, I don't have a previous > version of pypy already installed and am trying to create the first one.. > :( .. Assuming you are following "Building from Source": http://pypy.org/download.html#building-from-source - The link in ?2, "install dependencies" says: It is, of course, possible to translate with a CPython 2.6 or later [...] - ?4 says: works also with 'python' instead of 'pypy' IMO this is clear enough, but maybe you found another place where pypy cannot be replaced by python? -- Amaury Forgeot d'Arc -------------- next part -------------- An HTML attachment was scrubbed... URL: From yury at shurup.com Thu Oct 31 17:31:32 2013 From: yury at shurup.com (Yury V. Zaytsev) Date: Thu, 31 Oct 2013 17:31:32 +0100 Subject: [pypy-dev] Question on installing pypy and the source code .. In-Reply-To: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com> References: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com> Message-ID: <1383237092.2861.122.camel@newpride> On Thu, 2013-10-31 at 16:10 +0000, Dave Watson (davewats) wrote: > > I there a set of instructions for very first time builders ?? Hi, I think that you are misinterpreting the instructions on the website. They don't imply that you already have *PyPy* installed, but they do imply that you have *some* implementation of Python installed, such as CPython, which is probably already the case for you. So if you want to re-build PyPy completely, you need to translate it using some pre-existing Python interpreter, such as CPython, or PyPy itself, and then build it, for which you need a C compiler. Once you've got the PyPy package, you can use virtualenv, so that you can easily switch between different versions and install additional packages using pip. HTH! -- Sincerely yours, Yury V. Zaytsev From davewats at cisco.com Thu Oct 31 20:10:51 2013 From: davewats at cisco.com (Dave Watson (davewats)) Date: Thu, 31 Oct 2013 19:10:51 +0000 Subject: [pypy-dev] Question on installing pypy and the source code .. In-Reply-To: <1383237092.2861.122.camel@newpride> References: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com> <1383237092.2861.122.camel@newpride> Message-ID: <7265C53BF726F640BF92D1F05619FDE720181B72@xmb-aln-x11.cisco.com> Hi Yury .. I have just been looking at the pypy internals, and I have been trying to find out where I can add a new library directory for linking/includes, rather than depending on the system libraries, which in my case don't have the required dependent libs, so I will need to add them in order to get pypy to compile .. I was not able to find a convenient place to add in a additional Library path .. I found the variable LIB_ROOT, that may be possible to modify, but it didn't look like it was a great location to modify .. I was wondering I you or someone else might know of a good place that would allow the required alternative library location to be added to the configuration .. Thanks for any help that you can offer .. Cheers Dave W. -----Original Message----- From: Yury V. Zaytsev [mailto:yury at shurup.com] Sent: Thursday, October 31, 2013 12:32 PM To: Dave Watson (davewats) Cc: pypy-dev at python.org; Steve Marsh (stevmars); Gilbert Ramirez (gilramir) Subject: Re: [pypy-dev] Question on installing pypy and the source code .. On Thu, 2013-10-31 at 16:10 +0000, Dave Watson (davewats) wrote: > > I there a set of instructions for very first time builders ?? Hi, I think that you are misinterpreting the instructions on the website. They don't imply that you already have *PyPy* installed, but they do imply that you have *some* implementation of Python installed, such as CPython, which is probably already the case for you. So if you want to re-build PyPy completely, you need to translate it using some pre-existing Python interpreter, such as CPython, or PyPy itself, and then build it, for which you need a C compiler. Once you've got the PyPy package, you can use virtualenv, so that you can easily switch between different versions and install additional packages using pip. HTH! -- Sincerely yours, Yury V. Zaytsev From yury at shurup.com Thu Oct 31 23:05:44 2013 From: yury at shurup.com (Yury V. Zaytsev) Date: Thu, 31 Oct 2013 23:05:44 +0100 Subject: [pypy-dev] Question on installing pypy and the source code .. In-Reply-To: <7265C53BF726F640BF92D1F05619FDE720181B72@xmb-aln-x11.cisco.com> References: <7265C53BF726F640BF92D1F05619FDE7201819D4@xmb-aln-x11.cisco.com> <1383237092.2861.122.camel@newpride> <7265C53BF726F640BF92D1F05619FDE720181B72@xmb-aln-x11.cisco.com> Message-ID: <1383257144.2618.16.camel@newpride> On Thu, 2013-10-31 at 19:10 +0000, Dave Watson (davewats) wrote: > > I was wondering I you or someone else might know of a good place that > would allow the required alternative library location to be added to > the configuration .. I think that the easiest way to go is to manipulate CFLAGS and LDFLAGS directly. Just export them and they should be picked up automatically. If you have a specific problem, please post the raw error messages, otherwise, it's only possible to offer very generic suggestions. -- Sincerely yours, Yury V. Zaytsev From drsalists at gmail.com Thu Oct 31 23:29:22 2013 From: drsalists at gmail.com (Dan Stromberg) Date: Thu, 31 Oct 2013 15:29:22 -0700 Subject: [pypy-dev] pip install rbtree fail in pypy env! In-Reply-To: References: Message-ID: On Thu, Oct 31, 2013 at 1:53 AM, Armin Rigo wrote: > Hi KaShining, > > On Thu, Oct 31, 2013 at 9:32 AM, KaShining wrote: > > src/rbtree.c: In function '__pyx_f_6rbtree_6rbtree_byOffset': > > This is using Cython. Compiling Cython modules with PyPy > kind-of-works but is shaky. > Here's a pure python red black tree: https://pypi.python.org/pypi/red-black-tree-mod Full disclosure: I put it on Pypi after getting it passing pylint and such. -------------- next part -------------- An HTML attachment was scrubbed... URL: