Strange Hotshot problem...

[KefX]

Hi guys! I'm still a bit of a Python newbie, but regardless I decided to embed
Python into a plugin for a freeware (closed-source) Windows music program
called Jeskola Buzz. (Man, I can't believe how many people keep telling me
"Extend Python instead!" when clearly I can't do that in this case. If somebody
here tells me this, I will explode in an appropriately gory fashion.) It's a
great program, and it works mainly on plugins other people write. Well, I had
been working on a guitar synth plugin in the summer, which started to sound
good. Then I lost the code (I was trying to rewrite it anyway but I wanted to
keep it as reference.) So I began to think about ways to try to make it better.

I eventually hit on the idea of using a scripting language to do this. I
started designing a language with a simple grammar (I like doing this kind of
thing in a semi-compulsive manner, heh)...but I tossed that aside, worked on a
ROM hacking/translating project some...well, my ROM hacking project required me
to write a small script, so I decided it was finally time to learn Python.

Geez, I love this language. :)

Anyway, now it's a couple months later, translation project gets put on hold
for reasons outside of my control, so I'm trying the guitar synth again. And I
thought...why don't I use Python? I was told I could never get it to run in
real time (we're talking about DSP, after all). I was about to give up on
pursuing the idea when I heard about Psyco, which should help speed up
everything that isn't rewritten as a C extension. I decided I'll give that a
shot, no matter how ridiculous the idea seems. My experience with ridiculous
ideas is that sometimes they actually work. :)

Well, to make a long story longer, I got it running and it does produce sound
(not the sound I would like to hear from it, though!) But brokenness of the
sound suggested that it's running too freakin' slow, and it was: Buzz's CPU
display said my own plugin was hogging the whole CPU! Well, okay, it's not
hopeless yet, I can profile it and see what I can do about the bottlenecks.

Which finally leads me to my problem: I can't get Hotshot to work properly. I
can't fix a bottleneck if I don't know where it is! ;) Remember, I'm calling
various Python functions from C++. See, in my module's init code, I'm basically
I'm creating the hotshot.Profile object with the required path, bound to
_profiler, then I do _profiler.start() ...at the very end, I do
_profiler.stop() and _profiler.close(), and then I try to load the stats back
with hotshot.stats.load() with the same path. Well, it finds the file just
fine, but it can't read it. It raises an IndexError from trying to pop an empty
list or some such. I've tested Hotshot with simple test scripts in pure Python
and it works just fine. I thought Psyco might be the problem, but I commented
out all use of Psyco in my script and the problem still exists, so I think the
use of C++ to call Python code is messing it up.

Is this a bug in the Hotshot package? Is there a workaround? Oh, yes, I also
uploaded the offending profiler stats snapshot, in case you want to look at it.

http://members.aol.com/keflimarcvsx/hotshot.prof

If you open it in the interactive Python interpreter and try
hotshot.stats.load('hotshot.prof')  you should get the same result I do,
complete with traceback.

- Kef

P.S. Sorry for making this post so long :P