[Mailman-Developers] [GSoC 2012] Metrics

[George Chatzisofroniou]

The following document is the lowest level of my design concept. You
may also read it in my blog [1]. Of course, comments are very welcome.

--
Models

In order to store statistical data, the app will use some Django models:

Author

This model represents an author of the mailing list. It mostly keeps
track of the number of postings and number of threads started.  It has
the following fields:

- authorid – IntegerField
- authormail – CharField
- totalmails – IntegerField
- totalthreads – IntegerField
- firstmsgdate – DateTimeField
- lastmsgdate – DateTimeField

MailingList

This model  counts  the total number of postings and threads started.

- totalmails – IntegerField
- totalthreads – IntegerField

Month

This model associates the author and the mailing list with each month.

- author – ForeignKey
- month – CharField
- postscount – IntegerField
- threadscount – IntegerField
- mailinglist – Boolean (if this is true it corresponds to the whole
mailing list)

Year

This model is similar to month. It has a year field instead of a month field.

Views

To display the metrics the Django template system will be used. To
output the charts i will create some custom tags. The three following
views will be used:

- General page – On top, there will be general metrics about total
authors, total mails and total threads and below three charts (AJAX
based) that represent number of posts per author, number of threads
per author and mailing list’s yearly usage. Even below there will be a
number of charts (equal to the number of years of list’s existence)
that output monthly usage. At the end, there will be tabular data
representing the authors of the mailing list along with their number
of posts, number of threads started and the date of their last post.
The user will be able to order the tabular data (alphabetically,
ascending/descending on number of posts, number of threads, date of
last message) by clicking on the table’s headings (Mail, Mails Sent,
Threads Started, Last Message).

- Author page – Each user will have his own page with his own metrics.
On top, there will be the email of the author, number of posts, number
of threads started and the dates of first and last message. Below
there will be monthly usage charts for each year the user is
subscribed to the mailing list.
Django Admin page – A ‘Generate’ button will be added to the Django admin page.
Settings

The Django app should handle the following configuration parameters:

- Host – Message store data host
- Port – Message store data port
- Masking – A multi-state variable (None, abbreviated, full) for
masking email addresses at the results (we don’t want the emails to be
spammed)

Interface to the Mailman core

- Metrics class – When a new post is sent, the Metrics class will
receive it through the IArchiver interface. The Posts field of the
Mailing List model (as well as the the related rows on the Month and
Year models) will increase by one. If the author’s email is not in the
database, it will query the mailman core database with the email, grab
the author’s id and a new Author row will be created. Otherwise if the
author is already in the database, the Posts field and the two foreign
fields (Month and Year) will increase by one

- Generate class – When the ‘Generate’ button on the Admin page is pressed:
* The Django models will be initialized (the metrics will go back to
zero). A progress bar will inform the administrator that the operation
is being processed.
* All the messages of the archive will be parsed by performing a
direct Python call to the IArchiver. Another instance to the IArchiver
will grab any mails sent while the parsing is going on.
* The metrics will be generated from scratch.
* The administrator will be informed with a success message when the
process is over.

[1]: http://sophron.latthi.com/gsoc-mailman/

-- 
George Chatzisofroniou
sophron.latthi.com