[pypy-dev] funding/popularity?

[Dima Tisnek]

I sort of figured it out, although I don't have a ready solution.

This affects opera 11, stable ff, ff 4.0b7; amazingly it does not affect ie8.
Basically memory consumption of the plot appears to be proportional to
plot area.
Normal plot that you get by default at
http://speed.pypy.org/comparison/ costs about 100M of browser memory
consumption:
opera 130M, stable ff 70M, beta ff 90M at window size 1680x1050;
opera  80M, stable ff 55M, beta ff 70M at window size 1024x600;
Switching to "horizontal" produces a tall plot of same width and costs
about 300~700M of browser memory:
opera 720M, stable ff 370M, beta ff 370M at window wize 1680x1050;
opera 350M, stable ff 370M, beta ff 370M at window size 1024x600;

Suprisingly window size only matters while javascript produces the
plot, and not when window is resized, even though plot it resized with
the window correctly.

This alone is pretty heavy, but doesn't grind the browser.
What really grinds is that every time you change a tickbox on the
left, a plot is redrawn and another 200M of browser memory is wasted.
This is not double buffering, as next change adds yet another 200M or
so and so on, it appears that either js doesn't free something, or
browser caches or saves the previous page state.

As memory consumption grows, at some point browser hits the wall,
causes heavy swapping for some time, and I think garbage collection,
because practical (but not virtual) memory usage first drops to
something like 20~50M and then returns to "normal" 300M.
opera ~30 seconds, stable ff about a minute, beta ff several minutes
(total system mem 1G, cpu Atom @1.6GHz)

Perhaps OS also plays a role in the grind, as it is clearly swapping
and swaps out too much? or triggers gc too late and gc has to pull the
pages back from disk to perform collection?

ie8 doesn't use that much memory, as a matter of fact memory
consumption starts little (40M) and changes very little (only +10M) if
you go to horizonatal view; the price is very slow rendering, more
than 10 seconds per column change.

I'll post this on firefox bugzilla too, let's see if someone has a solution.

Meanwhile perhaps pypy speed center could start with a smaller plot
area (or fewer columns as that makes horizontal plot smaller) to
accomodate varying hardware and system mem usage that users might
have?
The simplest would be a warning next to "horizontal" checkbox.

On 21 December 2010 01:06, Miquel Torres <tobami at googlemail.com> wrote:
> Hi Dima,
>
>> another temp problem with speed pypy is that it's terrubly slow in ff
>> beta. it also occasionally grinds in stable ff and opera, but I guess
>> this can be forgiven for the sake of simplicity / developer effort.
>
> Well, speed.pypy is actually fast in all modern browsers. The problem
> you are referring to is probably caused by a bug in the javascript
> plotting library (jqPplot) that is triggered in the comparison view
> when there are some results with 0 values. It only appears for some
> plot types, but it is very annoying because it grinds the browser to a
> halt like you say. Is that what you meant?
>
> We are looking into it, and will fix that library if necessary.
>
> Cheers,
> Miquel
>
>
> 2010/12/21 Dima Tisnek <dimaqq at gmail.com>:
>> On 20 December 2010 19:21, William ML Leslie
>> <william.leslie.ttg at gmail.com> wrote:
>>> On 21 December 2010 11:59, Dima Tisnek <dimaqq at gmail.com> wrote:
>>>> More visibility for performance achievements would do
>>>> good too.
>>>
>>> Where are pypy's performance achievements *not* visible, but should be?
>>
>> for example http://shootout.alioth.debian.org/
>> doesn't say which pypy version is used, what options, doesn't have
>> performance figures for multithreaded/multicore
>>
>> also benchmarks are kinda small, most of them are not docuemented, and
>> I couldn't find any info if the same python code was used for cpython
>> and pypy (both shootout and speed pypy) or interpreter-specific
>> verions were used, that is each tweaked for best performance given the
>> known tradeoffs for each implementation.further the most standard
>> benchmarks, pystone, etc. completely ignore the fact that real world
>> programs are large and only a few small paths are execured often.
>>
>> another temp problem with speed pypy is that it's terrubly slow in ff
>> beta. it also occasionally grinds in stable ff and opera, but I guess
>> this can be forgiven for the sake of simplicity / developer effort.
>>
>> if you google for 'python performance' you don't get a single link to
>> pypy on the first page, as a matter of fact, codespeak is poorly
>> indexed, it took me quite some time to get some of my questions
>> answered with a search. also if you look up 'pypy gc' you get a page
>> on codespeak, but to interpret what the data actually means is so far
>> beyond me.
>>
>> a good overview is found in the mainling list
>> http://codespeak.net/pipermail/pypy-dev/2010q1/005757.html then again
>> slowspitfire and spambayes bits are outdated by now.
>>
>> the definitive good thing about pypy is that it's much easier to find
>> out about its inner workings than that of cpython!
>>
>> hopefully a bit more of end-user stuff get known.
>> let's call it pypy public outreach (feature request)
>>
>>>
>>>> Sidetracking... one day when pypy jit/gc/etc are all worked out, how
>>>> hard would it be to use same techniques and most of backends for some
>>>> unrelated language that doesn't have jit yet, e.g. php?
>>>
>>> You know that pypy already has implementations of other languages,
>>> right - Scheme, Javascript, Prolog, IO and smallTalk? They aren't
>>> integrated with the translated pypy-c, but they show that it is not
>>> too difficult to write a runtime for any dynamic language you choose.
>>
>> Oh I didn't know there were so many, and I mistakenly though that js
>> was a target, not implmented langauge. In any case I read somewhere
>> that js support was retired...
>>
>>>
>>>> And how hard
>>>> would it be to marry two dynamic languages, so that modules from one
>>>> could be used in the other? Or that modules written in rpython could
>>>> be used in several langs?
>>>
>>> It's in the "interesting problems" bucket, and the effort required
>>> depends on the level of integration between languages you want.  There
>>> are several projects already attempting to do decent integration
>>> between several languages, besides the approach used on the JVM, there
>>> are also GNU Guile, Racket, and Parrot, among others.  It might be
>>> worth waiting to see how these different projects pan out, before
>>> spending a bunch of effort just to be an also-ran in the
>>> multi-language runtime market.
>>>
>>> However, implementing more languages in rpython has the side-effect of
>>> propagating the l * o * p problem: it introduces more and more
>>> implementations that then have to be maintained, so good
>>> cross-language integration probably belongs /outside/ pypy itself, so
>>> existing runtimes can hook into it.
>>
>> Makes perfect sense, after all any given other language hardly has the
>> same data model as python.
>>
>>>
>>> But it would be an interesting experiment (to marry the various
>>> interpreters pypy ships with), if you wanted to try it.
>>>
>>> My two cents.
>>>
>>> --
>>> William Leslie
>>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>