[Mailman-Developers] Approach for Auto moderation system

[Richard Damon]

On 3/7/15 11:20 PM, Aanand Shekhar Roy wrote:
>> One concern here is that "Thread" is a fragile term in email. Unless you
>> are planning on some form of message body analysis to group messages
>> together, you are going to need to rely on the In-Reply-To and
>> References headers of the incoming email, which can have its
>> difficulties. If you are going to thread by something else, like the
>> subject, you may find people making minor changes in the subject to
>> bypass the moderation.
> Yes, I agree ,but having a system that curbs this problem will not again
> be a very good solution, for eg. A thread regarding discussion on mailman2
> can have the name of the thread "Mailman 2", and for mailman3, the thread
> will be named as "Mailman 3". Now ,it can also appear that someone has
> made a minor change to avoid moderation but it is not so.
>> First, these headers are optional, and some mail agents may not generate
>> them, and more importantly, the subscriber can bypass this linking by
>> creating a reply as a "new message" thus bypassing the auto moderation.
> Since I wish to implement this system as a plug-in it will be optional for
> the list admins and having an MTA that generates headers can be kept as a
> requirement.
It isn't the MTA that deals with the threading, but the MUA, and that 
can't be controlled by the list admin (as it is the USER agent), MUAs 
don't announce that they don't support them or that they were used in a 
way to bypass generating them.

My comment isn't that you can't create a system as you have defined it, 
but that there is a fundamental flaw in your design that will mean it is 
very easy for people to bypass it, making the add-on basically 
worthless. It will limit the posting of the people who follow the rules, 
but not those who figure out the holes and bypass the limit. The people 
who tend to follow the rules are rarely the problem.

>> Second, there is an unreliability in these headers as they will not
>> necessarily reference the "start" of the thread, but may only list
>> messages later in the thread, and to get your "Thread Name", you are
>> going to have to keep a full history (for some period back) of messages
>> and what thread you determined them to be in to figure out what thread
>> this message is in.
> What we can do is to remove general keywords like "Re:[ ]" and "Fwd:[ ]"
> from the thread and then add to the database, so we don't need to store
> all the history, just information about last one does the job, we check
> that through Table1.
Ok, this make it clear that you are not working on "Threads" as defined 
by the reference headers, but something similar that only uses the 
Subject header, which is MUCH more prone to being "gamed" by minor 
tweaking that might not even be noticed without close looking.

(Perhaps you don't understand the real concept of threading as your 
email client doesn't support it, I see my replies attached to your 
messages, slightly indented, while your reply to me became a "new 
thread" as your client didn't indicate what it was a reply to. You can 
see this same effect in the lists threaded archives).

This is why I say that having only "per-thread" limits isn't workable, 
as threads are too fragile. If you need to have a per-thread limit you 
need to also either limit total-posts or new-thread posts.
>> This means that any system that ties to limit the rate "in thread" must
>> also have a similar (but perhaps different value) limit on total
>> postings or creation of new threads.
> We do not aim at limiting threads or posts on a thread we are just slowing
> it down to provide room to other users. If we decrease no of posts on a
> thread it might affect the discussion going on as some threads are very
> long and also important and can't be shortened.
>
> Regards
> Aanand
>

Yes, I understand that the limit isn't a limit on absolute number of 
posts, but a "rate-limit" on posting.

-- 
Richard Damon