[Python-Dev] s/hotshot/lsprof

[Jeremy Hylton]

Here's another attempt to disentagle some issues:
- Should lsprof be added to the standard distribution?
- Should hotshot be removed from the standard distribution?

These two aren't at all related, unless you believe that two is the
maximum number of profiles allowed per Python distribution.

I've never trusted results from hotshot, but I'd rather see it fixed
than removed.

Jeremy

On 11/21/05, Brett Cannon <bcannon at gmail.com> wrote:
> On 11/21/05, Armin Rigo <arigo at tunes.org> wrote:
> > Hi Martin,
> >
> > On Mon, Nov 21, 2005 at 08:12:53AM +0100, "Martin v. L?wis" wrote:
> > > If bugs are in the heavily-used parts of the library, like regular
> > > expressions, it doesn't matter much if the original author goes
> > > away for some period of time - other contributors will fix the bugs
> > > that they care about, and not by rewriting the entire thing.
> >
> > I see no incremental way of fixing some of the downsides of hotshot,
> > like its huge log file size and loading time.  I doubt people often find
> > the motivation to dig into this large orphaned piece of software.
> > Instead, they rewrite their own profilers, because writing a basic one
> > is not difficult.  It is much less difficult than, say, writing a basic
> > regular expression engine (but even the latter has gotten rewritten at
> > times) -- unless you want to go into the advanced corners mentioned by
> > Tim.
> >
> > Some guys posted their 'lsprof' on SF because it was well-polished and
> > they found it useful, so here I am, arguing for a standard library
> > containing preferably simple pieces of code that work and are practical
> > for the common advertised use case.  I'm not even sure in this case why
> > we are arguing: the new piece of code's interface can be made 100%
> > compatible with the documented parts of the previous interface; the
> > previous module has been around for longer but so far it produced
> > half-meaningless numbers due to bugs.
> >
>
> Just because it is starting to feel like the objections are getting
> spread out amongst various parts of this thread, I want to try to
> summarize them as I remember them and give my input on them.
>
> So one objection seems to be the question of maintenance.  Who is
> going to keep this code updated and running?  As has been pointed out,
> Hotshot is not perfect and its development basically stopped.  So
> people being a little on edge about yet another profiler that might
> not be maintained seems reasonable.
>
> But this worry, in my mind, is alleviated since I believe both Michael
> and Armin are willing to maintain the code.  With them both willing to
> make sure it stays working (which is a pretty damn good commitment
> since we have two core developers willing to keep this going and not
> just one) I think this worry is dealt with.
>
> The other issue seems to be some people wanting to keep Hotshot around
> for a few releases until lsprof can prove its worth.  I believe this
> is what Barry is asking for.  Now Armin has said that a wrapper around
> lsprof can be written that will match Hotshot's public API so its need
> is not there if lsprof works and the wrapper is good.
>
> If it wasn't Armin or someone else whose opinion I trusted, I would
> say go ahead and keep Hotshot around and then eventually do the
> wrapper.  But since it is Armin making this claim and the PyPy team
> uses this thing (who has several members who I think know what they
> are doing  =)  I have faith in them coming up with a good wrapper.
> Thus I say removing Hotshot is fine.
>
> Lastly, there is the argument of whether we should even include a
> profiler.  Personally I say yes.  It is another battery that is rather
> nice.  I think if the profiler finally had a good reputation of being
> accurate and useful it would get more play in the real world.  Plus we
> already include other development tools such as IDLE with Python so it
> seems fitting to include other dev tools when we have the code and a
> maintenance commitment.
>
> In other words, I say let Armin and Michael add lsprof and the
> wrappers for it (all while removing any redundant profilers that they
> have wrappers for) with them knowing we will have a public stoning at
> PyCon the instant they don't keep it all working.  =)
>
> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>