[SciPy-user] maxentropy

[Ed Schofield]

On 31/03/2006, at 3:55 AM, Matthew Cooper wrote:

> I went through the new conditionalexample_high_level.py and I still  
> think there a small change that needs to be made (I think it's  
> small anyway).  I think that we want F to be the size
>
> F = sparse.lil_matrix((len(f), numcorpus*numsamplespace))
>
> where numcorpus = len(corpus)
>

Okay, this seems straightforward.  I've changed the example so there  
are only columns of F for contexts that appear in the corpus.

> I don't think this alters your code, as long as the pmf and F  
> matrices are initialized correctly.
>

Yes, you're right.

> At test time, we do need to evaluate the feature functions on  
> unseen documents, but this can be handled more easily.
>

I'm not sure how yet.  I'll give this some thought.


> I have another question.  I haven't installed your version of scipy  
> outright since it was a bit of a pain to get the current stable  
> distribution up on my machine.  However, if I need to load a bunch  
> of modules from your version to test the conditional models is  
> there an easy way to do that?

Which scipy version are you using?  If it's recent enough, you can  
just copy my maxentropy.py and sparse.py files over the installed ones.

I'm happy enough that it works now; I've merged the new sparse  
functionality back into the trunk, and I'll do the same with  
conditional maxent class in the next few days.


>   At the moment, I couldn't import sparseutils (I can't find  
> the .py file since I probably haven't built it?).

sparsetools is written in FORTRAN, with an f2py interface, so it  
needs to be installed properly by numpy.distutils.  But sparsetools  
is the same in my branch as in the trunk ...

-- Ed


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20060403/aa823022/attachment.html>