ANN: Veusz 0.5 - a scientific plotting package

[Jeremy Sanders]

On Mon, 18 Apr 2005 13:40:09 -0700, jdh2358 wrote:

> I'll start by saying that I for one won't criticize you for rolling you
> own plotting package rather than join forces with an existing project.
> I've been accused of the same, on more than one occasion :-)  But I'm also
> aware of the problem that this creates -- the tyranny of choice. python is
> so fun to code in that many developers are looking for a reason to find an
> existant package inadequate so they have an excuse to write their own
> replacement.  Hence we have a proliferation of web-app frameworks,
> plotting packages, array objects and so on.  There is a lot of duplicated
> effort in many arenas and it would be nice to collaborate more.

True. It's sad that it's just more fun to go off and write something
yourself, but it is fun :-)  I'm afraid I'm not very good with using other
people's codebases.

> I read over your scipy list of problems that you found in matplotlib --
> some were helpful and some, as you note, have been long fixed.  One
> critique you might flesh out for me is the notion that matplotlib's object
> model is baroque -- most of the developers feel the object model is fairly
> solid.  You weren't by chance, trying to use the procedural pylab (aka
> matlab) interface, were you, since pylab itself is just a wrapper of the
> OO matplotlib API?  One area in the object model that we plan to change is
> to make high level plot objects (scatter, error, histogram) etc, proper
> objects, ala gnuplot.  Right now they are methods that generate primitive
> objects (lines, rectangles, text, etc).

I thought I was using the object interface. It seemed strange to me that
methods of the axes were used to plot data, draw legends, and so on... It
seemed to make much more sense to have these as objects themselves. The
main problem is that there's no object you can alter to change their
appearance.

I quite like the object system I've developed, where the external
interface is completely based on building the object hierarchy and setting
properties of the objects (plus there is a "functional" interface, but
this is only used rarely for active operations, like fitting data).

I wanted an interface where I could twiddle a bit, and change the axis
from log to linear, or vertical to horizontal... I wasn't sure which
variables in the matplotlib source I could touch and get away with in
future releases :-)

> Another area you identify as a problem with matplotlib is the need to
> regenerate the entire graph when one property is changed.  This is true in
> one way and false in another.  matplotlib does have a proper object model
> -- eg, you can selectively change the facecolor of a marker w/o
> regenerating the graph scene.  But the *drawing* hierarchy (as opposed to
> the object hierarchy) needs some work .  Once you have changed a property,
> the entire graph is redrawn.  This is a known limitation and will change
> in the not-too-distant-future.  One advantage of working in mainstream in
> open source software is the network effect.  With 10 some-odd developers
> including institutions such as the U of C, STScI, JPL and NOAA, bugs and
> limitations of matplotlib tend to be fixed within minutes, days, weeks or
> months.

It looked to me that you could change some properties (like line style)
after creating the graph, but other things were hard to change (marker
style?). Therefore I deleted the graph and rebuilt it from scratch each
time. The way you modify a property is different from how you set it when
building a graph, and so it was difficult to create a transparent
interface.

> Your package looks very nice.  It specifically addresses two limitations
> in matplotlib that we would like to address -- a GUI interface for
> creating figures and a way to save the figure at any point as a high level
> description (rather than an image).  I do wish you had publicly voiced the
> problems you ran into along the way; I just searched the archives and saw
> only one post from you on the users list which I answered 28 minutes later
> with
> 
> http://sourceforge.net/mailarchive/message.php?msg_id=10124206
>
> after which I never heard from you again.  Such response times are fairly
> typical on the list, but if you don't report the bugs and follow up on the
> suggested fixes, we can't fix them.

Sorry - much of my work was done when I didn't have a very good internet
connection, and so it was hard for me to collaborate over bugs. I took the
easy option of expanding the small codebase I already have (which I knew
pretty well!), over debugging someone else's code.

> Anyway, nice work on veusz.  Are you committed to the GPL license?
> matplotlib uses a more permissive license (PSF compatible) mainly to
> encourage contributions from the commercial sector.  As you suggest, it is
> still possible for someone to take the work you've done on the GUI
> frontend and expose matplotlib as a backend based on your prior
> experiments.  The NASA Jet Propulsion Laboroatory and others are
> supporting the QT backend, and they might be willing to contribute some
> time to porting your GUI frontend to matplotlib, provided the license of
> veusz was compatible with the mpl license.

I quite like the idea of the GPL licence, as changes to the codebase
remain public. I'm a bit hampered by the GPL licence on PyQt to change my
code to another licence. I'm happy to re or dual-licence my code for other
projects if they believe there isn't a problem with linking PyQt against
their code. 

It may not be too hard to wrap around matplotlib with my current setup.
You'd just need to create objects to represent each plottable thing, and
would wrap to the matplotlib plotting methods of the Axes.

> There are two specific issues you raised in the scipy post you linked to
> that I'll take issue with:
> 
>   >> In my experience, the plotting code isn't that hard to do. The  
>   >> hardest part was making the axes look good.
> 
> In my experience, the hardest part of plotting code is supporting all the
> ways in which people want to use the code, and that includes embedding the
> plot in their GUI of choice, working across platforms, supporting script
> based, interactive shell work, application developers,  web app developers
> and so on.  There are quite a few plotting idioms that are mission
> critical to many developers, and supporting them all is a complex task. 
> I'm fond of pointing out the combinatorial problem -- take every GUI,
> cross two major versions in the wild, cross the big-three platforms, cross
> the standard ways in which people want to work with their plots and you
> have a lot of work to do.  veusz "solves" this problem by restricting
> choice, which is a respectable solution.  matplotlib takes the opposite
> tack and tries to support the user in the environment in which they want
> to work.

Yes - it's much harder to do the general thing like matplotlib does. Of
course you get the end benefit which is flexibility.

I do restrict people to PyQt of course. It was a route I chose to get a
quick result.

> The second point is
> 
>    >> I've done 90% of what's needed there. Basically, I'll be happy when
>    >> contouring and images are added.
> 
> I tend to disagree that contouring and images are only 10% --  good
> contouring, contour labeling, and image support strong enough to satisfy
> people for whom images are their lifeblood is hard, and almost impossible
> to do efficiently in pure python.  matplotlib isn't there yet, but the
> support for both is pretty strong and getting better, and matplotlib is
> increasingly being used by astronomers for image display.
>  But I see from your web page that you too are an astronomer, so I
> expect that you will handle these areas nicely; again, too bad we couldn't
> collaborate more. Another tough area to do right of course is nonlinear
> transformations (polar, log, mapping projections) with the associated
> problems of proper tick locating and labeling...

Maybe I glossed over the coordinate transformation thing! Yes - doing
proper sky coordinates and things like that is hard in general, in
particular getting the tick marks correct.

I had an interesting time getting it working for linear and log
coordinates, but that was well before I had heard of matplotlib! I think
my current code isn't bad for log and linear.

It's nice to do the transformation with numarray in a single line!

I took a long time to decide whether to continue with my own plotting
routines, or use matplotlib, and I found it a hard decision. I went my own
route as I had the freedom to redesign as I liked, was having fun with it,
I had written a fair chunk already, and I knew exactly how it worked. I'd
also got slightly annoyed by the bugs in previous versions of matplotlib
(which is a lot better in recent releases, I'm sure, and my code is not
free of bugs!).

Thanks for your reply

Jeremy