[Mailman-Developers] Gsoc Support for Message queue based email archiver

[Anirudh Dahiya]

Hello
I have been getting to know more about the message queues and the pub/sub
systems available.
As I am trying to formalize the project description for the proposal, I
have a few doubts regarding this.

1. Reliability - Currently, as I see, the messages are archived in
Hyperkitty via a simple POST request, but in the event of some failure(like
hk server being down for a while), there is no way the message is being
tried to be archived again. Thus are we aiming to add some reliability to
the system via the message queue systems?
In our use case, I believe reliability is a critical issue as we make
threads from individual mails in the archiver.
Apart from this and extensibility, what all advanteges are we looking
forward to as a result of this project?
In Zeromq, I am still searching for a storage based message queue with
multiple consumers. Also, the pub/sub model is based around the idea of
message being dropped (as in subscribers not listening to the broadcaster
miss out on the messages.)
I found an interesting approach based in redis
https://davidmarquis.wordpress.com/2013/01/03/reliable-delivery-message-queues-with-redis/

2. Should I add/mention the potential scope for adding mailman events (like
list creation etc) to our archivers?

3. Am I supposed to alter Hyperkitty(as what seems to be mentioned in the
project description) or should the basic scope of the project stay about
building the pub/sub or message queue interface(as what seems to be
mentioned by some of our conversations on irc)

I do partially know answers to some of the questions above, but would like
a confirmation and a concrete idea about the project before I apply my
proposal.
Thank you for your time
Regards
Anirudh Dahiya
(irc spark)

On Tue, Mar 15, 2016 at 3:00 PM, Florian Fuchs <f at florianfuchs.com> wrote:

> On Sun, Mar 13, 2016 at 16:22:25PM +0530, Anirudh Dahiya wrote:
>
>> Hello
>> I am a computer scrience undergrad at IIIT-Hyderabad, India.
>>
>> I was looking through the GSoC 2016 ideas and would like to know more
>> about
>> the project "message queue based email archiver"
>>
>
> Great to hear you're interested in this project. I've just responded to
> another question regarding this project that is equally relevant to you:
>
>  http://www.mail-archive.com/mailman-developers%40python.org/msg16305.html
>
> I have played with codebase of mailman and postorius for some time now and
>> am currently trying to understand how Hyperkitty works.
>> Also, as suggested to other students regarding this project, I have gone
>> through the chapter on mailman in the book The Architecture of Open Source
>> Applications, the pycon talk on mailman by Barry and have also gone
>> through
>> relevant parts of the documentation.
>>
>
> Sounds good. If you're already reading through the HyperKitty source, you
> might give special attention to its implementation of Mailman's IArchiver
> interface, as well as the implementations for the mail-archive.com and
> mhonarc archivers in the Mailman core projects.
>
> Florian
>
>
>> Regards
>> Anirudh Dahiya
>> (spark on IRC)
>> _______________________________________________
>> Mailman-Developers mailing list
>> Mailman-Developers at python.org
>> https://mail.python.org/mailman/listinfo/mailman-developers
>> Mailman FAQ: http://wiki.list.org/x/AgA3
>> Searchable Archives:
>> http://www.mail-archive.com/mailman-developers%40python.org/
>> Unsubscribe:
>> https://mail.python.org/mailman/options/mailman-developers/f%40florianfuchs.com
>>
>> Security Policy: http://wiki.list.org/x/QIA9
>>
> _______________________________________________
> Mailman-Developers mailing list
> Mailman-Developers at python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives:
> http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe:
> https://mail.python.org/mailman/options/mailman-developers/anirudhdahiya9%40gmail.com
>
> Security Policy: http://wiki.list.org/x/QIA9
>