[SciPy-User] visual inspection of kernel density estimators in 2d

[josef.pktd at gmail.com]

On Thu, May 26, 2011 at 4:31 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, May 26, 2011 at 15:17,  <josef.pktd at gmail.com> wrote:
>> I'm trying to find a visual way to see whether a kernel density
>> estimator "looks" good in 2d. For univariate it is easy to compare
>> histogram and the estimated density.
>>
>> Does anyone know what graphs would give a good visual inspection?
>>
>> My attempt with contours, which should show some oversmoothing if the
>> default gaussian_kde is used on mixture distributions:
>>
>> http://picasaweb.google.com/josef.pktd/Joepy#5611119163809333922
>
> With a bit of tweaking, that would work fine. A few suggestions:
>
> 1. Use smaller dots for the data, and maybe less color.
> 2. Use fewer contour lines, without labels. Maybe just make the
> contours that would have labels be a bit thicker so you can make a
> good apples-to-apples comparison between the two sets of contours.
> 3. Make the true contour lines gray and in the background.
> 4. Make the estimated contour lines black and in the foreground. I.e.
> draw the dots, then the true contours, then the estimated contours.

Thanks Robert,

I still have to figure out how to do many of these things with
matplotlib, for example I have to find the manual to change the line
width.

here is some improvement in this direction
http://picasaweb.google.com/josef.pktd/Joepy#5611180514445456594

here is a first attempt at 3d
http://picasaweb.google.com/josef.pktd/Joepy#5611133359567574274

This actually shows the oversmoothing quite clearly.

>
> You might also try a colormapped image plot of the difference between
> the two densities. Plot the data as small points (i.e. with 'k,') over
> the residual image.

just to see how it looks I tried the color overlay of the difference
on the same graph . It's a bit weird but quite informative.

http://picasaweb.google.com/josef.pktd/Joepy#5611180522655961714

(Given the nice colors in the last graph, I will try for a Miro next. :)

Since with real data we don't have the true pdf, I might try for a
histogram contour (?) as comparison.


Josef

>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
>   -- Umberto Eco
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>