[Mailman-Developers] RE: Mailman developer?

[Damien Morton]

We can use GIFs, but we cant use the LZW compression option. As the GIFs
we produce will be small, this is hardly a problem. At any rate, any of
GIF, PNG or JPEG will do. 

The advantage of using encryption rather than hashing is that the email
renderer needs to know nothing about the system except what it is given
in the querystring (and the key, of course). Using md5 hashes would be
more secure, but would require more information to be shared between the
pipermail cgi and the renderemail cgi; that is, a mapping of hashes to
email addresses. Frankly, I cant see email harvesters going to the
trouble of cracking encryption - no matter what kind.

I can think of an alterative technique for preventing email harvesting;
one that preseves the clickability of emaila addresses. It involves the
use of javascript. Using it, an email link would look something like
this:

<script language="javascript">
	email = decode("92731602eba1aa4f506604f8c3671ed83ea9");
	document.write("<a href='mailto:"+email+"'>"+email+"</a>");
</script>


decode() is some kind of decrypting function. Possibly something as
simple as a substitution cipher.

The disadvantage of this technique is that email addresses wouldn't be
accessible to people using a non-javascript capable browser, and browser
variations would tend to make this less reliable than the image viewing
technique. For example, I have found that Netscape is less than reliable
when it comes to document.write().

One solution to the non-javascript browser issues would be to use the
<noscript> tag to deliver the email address as an image.

<noscript>
<img src="/render-email.py?92731602eba1aa4f506604f8c3671ed83ea9">
</noscript>


One problem with all this is that pipermail renders HTML as it stores
emails, rather than as they are viewed. The issue here is that the
original emails are lost; if we encrypt (or hash) email addresses, then
losing the key (or hash->email mapping) implies also losing the
addresses. There are some comments in the pipermail/HyperMail source
code about rendering to html on viewing rather than storage, but
converting to this scheme would imply a backwards compatability issue:
how to import emails alerady rendered to html.

A backwards compatable solution would be to render to html twice - once
on storage and again on viewing. The viewing renderer would be
responsible for encrypting/obfuscating email addresses.

Im going to join the mailman-developers list with a temporary email
address. Get some more input on this issue. 

-----Original Message-----
From: Barry A. Warsaw [mailto:barry@zope.com] 
Sent: Wednesday, 30 January 2002 21:18
To: Damien Morton
Subject: Re: Mailman developer?



>>>>> "DM" == Damien Morton <Damien.Morton@acm.org> writes:

    DM> 	Im under the impression that you are one of the main
    DM> developers and/or maintainers of Mailman. I hope you don't
    DM> mind me writing to you.

Nope.

    DM> 	I notice that Mailman obfuscates email addresses to a
    DM> certain extent, but replacing the @ symbol with %40 or
    DM> &atmark; is hardly sufficient. An even vaguely intelligent
    DM> email harvester will see through this.

True.

    DM> 	The feature im proposing is to render out all email
    DM> addresses in the archive as GIFs. I would have pipermail
    DM> render out <img> tags whose src is an encrypted version of the
    DM> email address. A companion CGI script would decrypt the email
    DM> address and render it out as a GIF image using PIL or
    DM> somesuch.

Of course, because Mailman is a GNU project, we can't use gifs, but pngs
or jpegs would work just as well.  IIRC, PIL can generate either of
those formats.

    | Instead of rendering this:
    | <a href="mailto:your.email@address">your.email@address</a>

    DM> You'd render this instead: <img
    DM> src="/render-email.py?92731602eba1aa4f506604f8c3671ed83ea9">

    DM> 	This exmaple uses the simple rotor encryption that
    DM> comes with python and the key is "the quick brown fox jumped
    DM> over the lazy dog"

I usually use the md5 module to generate unique keys in such situations.

    DM> 	Ive been looking at pipermail, and it is
    DM> _ugly_.

Tell me about it!  About it's only saving grace is that it's reasonably
well integrated and it's all in Python.  Other than that... ;)

There have been lots of discussion over the years about ditching that
code and doing it right, so I'd suggest pouring over the
mailman-developers archives (yes, you'll miss being able to search it
;).  It's a lot of work though, so it currently languishes for lack of a
motivated champion.
    
    DM> Nonetheless, I'm fairly sure I can add this functionality
    DM> easily. Im running w2k, however, and I see that Mailman isnt
    DM> really meant for w2k.

Correct.
    
    DM> As I would be working on the pipermail part of mailman only,
    DM> it might be easier to get only that component up and running
    DM> under widnows. Not sure if theres a sample archive that comes
    DM> with mailman, but... any suggestions welcome.

Nope, but you can of course grab the raw mbox for any archive, or for a
month of messages.  Note that that does point to another source of
leaked addresses, one that won't be directly affected by your idea.
However I could see hiding raw mbox access behind a cgi POST which
should effectively stop today's harvesters.

    DM> 	The downsides of this functionality are that it might
    DM> incur a performance penalty and that it eliminates the
    DM> clickable mailto: functionality persently there.

System-wide caching should alleviate the performance hit (modulo cgi
overhead).  The loss of clickable mailto: would be a drag, although I
don't know how much it would be missed in practice.

    DM> 	As far as functionality goes, I imagine that the bulk
    DM> of any mailman bandwidth will be from spiders, and these are
    DM> unlikely to traverse an image source link. Secondly, simple
    DM> caching should be very easy to implement.

    DM> 	As far as the clickable mailto functionality goes, I
    DM> have two suggestions. The first is that the
    DM> render-emails-as-images functionality could be a personal
    DM> preference of the sender of the email, and the second is that
    DM> that preference could be overridden by acquiring a cookie
    DM> through some bot and spider proof mechanism. I like the 'type
    DM> what you read in the image above' mechanism for detecting
    DM> humans.

I wouldn't make it a option of the sender, but of the list (or maybe
just of the site).

Anyway, it's a neat idea.  Mailman-developers would be the best place to
discuss it, but that does present a bit of a catch 22 for you. ;)

>From a practical standpoint, MM2.1 will likely go to beta this weekend,
meaning feature freeze.  However I encourage you to follow through and
work out some patches, if you'd be willing to assign copyright to them
to the FSF eventually.  Post any patches on Mailman's SF project page
and that will let other interested parties download it and test it out,
etc.

Cheers,
-Barry