[SciPy-User] issue estimating the multivariate Polya distribution: why?

[Emanuele Olivetti]

On 03/04/2012 06:54 AM, Aronne Merrelli wrote:
> On Sat, Mar 3, 2012 at 5:51 PM,<josef.pktd at gmail.com>  wrote:
>> On Sat, Mar 3, 2012 at 6:24 PM,<josef.pktd at gmail.com>  wrote:
>>> 3.3941765165211696e-23
>>>
>>> For larger values it seems to work fine, but it deteriorates fast when
>>> the loglikelihood drops below -15 or so (with the versions I have
>>> installed).
>> just an observation
>>
>> with iterations=1e7 I get much better numbers, which are still way
>> off. But I don't see why this should matter much, since you are
>> simulating alpha and not low probability events. (unless lot's of tiny
>> errors add up in different ways)
>>
>> Josef
> I'm a little out of my field here, so take this with a grain of salt.
>
> I think Josef's observation is the key; the problem is the number of
> samples in the MC is too low. This distribution seems very, very
> skewed;
[...]

> If you plot test_logmeans, it clearly shows a negative bias (relative
> to the analytic prediction) that decreases as the sample size
> increases.

Thanks Josef and Aronne, your tests and comments are very
useful. Indeed the distribution of interest is very skewed and
the specific nature of the skewness could be the explanation
of the unexpected behavior of the montecarlo estimate.

I am trying to set up a minimal and straightforward example
where same surprising effect will (hopefully) appear. I guess
that crafting a very skewed distribution - even a very simple
one - should show such an anomalous behavior and give insights.

Unfortunately until now I was not successful. But I'll let you
know as soon as I make some progress. Of course any help
in digging more on this issue is warmly welcome!

Best,

Emanuele