[Mailman-Developers] UI for Mailman 3.0 update

[Cristóbal Palmer]

Before getting into this (long) reply, I want to re-emphasize that
what I want is (1) the ability to plug in existing CAPTCHA systems
(notably reCAPTCHA) quickly and easily, and change simple config
settings to enable those CAPTCHAs for parts of the interface that have
been tested and confirmed to make sense. And (2) I want other
(non-CAPTCHA, probably transparent to most users) changes to some
points (eg. new user signup, moderator login) in the default MM
interface that are designed to reduce abuse. I want (1) merely to the
extent that it helps with (2). With that, more reply below.

On Tue, Jun 15, 2010 at 01:15:47PM +0900, Stephen J. Turnbull wrote:
> "Not every three-line patch needs to be a standard feature."  Or
> 300-line patch, for that matter.  But some do.  Are captchas a feature
> that ordinary Mailman users need?  Or are they something that "if you
> know enough to know why you need them, you know enough to code an
> appropriate Handler"?  (Or snaffle one from the CheeseShop, for that
> matter.)  I have my opinion ;-), but I'm willing to be corrected. :-|

While I could in theory maintain a patch, I have a lot of machines to
herd, and I am unlikely to customize anything unless I must do so in
order to meet a requirement. I very much want upstream to maintain
anything that is reasonable and I will configure within the parameters
set by them. This is not an issue of my expertise. This is an issue of
maintainability and good use of admin time.

> I subscribe to a *lot* of Mailman lists.

So do I, but neither of us can claim (as individuals) to be
representative of mailman users (subscribers, moderators, list owners,
etc.), and in this case our claims seem to clash, so we need to look
elsewhere.

> I would be mildly annoyed if
> uninformed list owners started using captchas just because they're
> easy to configure and because a lot of big names use them.  At this
> point, I don't see a good reason to make it easy to annoy millions of
> subscribers that way.

While I share your distaste for bad implementations of... anything,
really, CAPTCHAs do not annoy me unless they are poorly implemented,
and they do not have to be poorly implemented. If you're going to say
that millions of subscribers are going to be annoyed, please cite some
studies. Here, I'll start:

(As an aside, please contact me off-list if you're not able to access
the full text for one or more of the papers cited below, and I will
arrange for you to have access.)

(1)

Yan, J. and El Ahmad, A. S. 2008. Usability of CAPTCHAs or usability
issues in CAPTCHA design. In Proceedings of the 4th Symposium on
Usable Privacy and Security (Pittsburgh, Pennsylvania, July 23 - 25,
2008). SOUPS '08, vol. 337. ACM, New York, NY, 44-52. DOI=
http://doi.acm.org/10.1145/1408664.1408671

ABSTRACT

CAPTCHA is now almost a standard security technology, and has found
widespread application in commercial websites. Usability and
robustness are two fundamental issues with CAPTCHA, and they often
interconnect with each other. This paper discusses usability issues
that should be considered and addressed in the design of
CAPTCHAs. Some of these issues are intuitive, but some others have
subtle implications for robustness (or security). A simple but novel
framework for examining CAPTCHA usability is also proposed.

(2)

Bigham, J. P. and Cavender, A. C. 2009. Evaluating existing audio
CAPTCHAs and an interface optimized for non-visual use. In Proceedings
of the 27th international Conference on Human Factors in Computing
Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New
York, NY, 1829-1838. DOI= http://doi.acm.org/10.1145/1518701.1518983

ABSTRACT

Audio CAPTCHAs were introduced as an accessible alternative for those
unable to use the more common visual CAPTCHAs, but anecdotal accounts
have suggested that they may be more difficult to solve. This paper
demonstrates in a large study of more than 150 participants that
existing audio CAPTCHAs are clearly more difficult and time-consuming
to complete as compared to visual CAPTCHAs for both blind and sighted
users. In order to address this concern, we developed and evaluated a
new interface for solving CAPTCHAs optimized for non-visual use that
can be added in-place to existing audio CAPTCHAs. In a subsequent
study, the optimized interface increased the success rate of blind
participants by 59% on audio CAPTCHAs, illustrating a broadly
applicable principle of accessible design: the most usable audio
interfaces are often not direct translations of existing visual
interfaces.

And finally, my favorite: (3)

M Motoyama, K Levchenko, C Kanich, D McCoy, GM. Re:
CAPTCHAs–Understanding CAPTCHA-Solving Services in an Economic
Context. Proceedings of the USENIX Security Symposium, Washington,
D.C., August 2010. http://cseweb.ucsd.edu/~savage/papers/UsenixSec10.pdf

ABSTRACT

Reverse Turing tests, or CAPTCHAs, have become an ubiquitous defense
used to protect open Web resources from being exploited at scale. An
effective CAPTCHA resists existing mechanistic software solving, yet
can be solved with high probability by a human being. In response, a
robust solving ecosystem has emerged, reselling both automated solving
technology and real-time human labor to bypass these
protections. Thus, CAPTCHAs can increasingly be understood and
evaluated in purely economic terms; the market price of a solution vs
the monetizable value of the asset being protected. We examine the
market-side of this question in depth, analyzing the behavior and
dynamics of CAPTCHA-solving service providers, their price
performance, and the underlying labor markets driving this economy.


So I'm going to disagree with your premise that CAPTCHAs are
necessarily annoying to most people unless you give more than
anecdotal evidence that that is the case, and I'm going to disagree
that they are always or even usually useless for protecting parts of
WUIs.

> (1) it should be configurable per list (and off by default);

Agreed.

> (2) it should need to be enabled by the site admin (and off by
>     default);

Agreed, but only to the extent that having it available by default
would add a dependency, which is too much of a burden on the MM team.

>     The rationale for this is not just to make it harder to use the
>     feature, but that the site admin is likely to be more expert in
>     general to understand the limitations of the feature, and also
>     some of the benefits and costs accrue to the site rather to the
>     list community, so the site admin should have some input.

Definitely agreed.
 
> (3) both configuration tools should have documentation indicating that
>     captchas do not provide security; what they do is chase off the
>     frivolous (both bona fide users and would-be abusers).  This is a
>     genuine benefit in several ways for many lists; it's just not real
>     security because serious abusers will get through.

Disagree. This is like saying that putting a $30 (USD) cable lock on
my bike is not security because serious thieves could defeat it with a
large pair of bolt cutters. Mind you, I use a ~$100 (USD) chain lock
on my bikes, but that doesn't mean the $30 (USD) cable lock is
useless, especially if the replacement cost of your bike is $150
(USD). You seem to think that only $100 (USD) chain locks are worth
the effort, and that I'm insisting people use cheap locks. That is not
the case.

Furthermore, I think we may be in part talking past each other because
you have seen lots and lots of poorly-done CAPTCHAs, and the entire
concept has been spoiled for you by those bad implementations, and you
picture us wanting one of those bad implementations on by default in
mailman. That is not the case at all; it is also a straw man. CAPTCHA
systems (and services such as reCAPTCHA) have improved a lot in the
past three years, and nobody wants even the best of these to be used
in a silly way within the default MM3 WUI.

What I want is the ability to flip a switch and have CAPTCHAs
available to me, and then have switches in one or more places
(eg. moderator login, user signup) for those CAPTCHAs to be used every
time or after the Nth attempt, for example.

As I said before, there are several non-CAPTCHA approaches that I'd
like to see used by default, too. For example, forcing signups to
include a NONCE, rate limiting signups, etc. I don't want to get too
hung up on CAPTCHAs in particular, but I also don't want us to
completely reject them, since they are in fact useful and good if used
properly. I'm very sorry you dislike them so much and have had bad
experiences with them, but please let's have a more scientific
discussion of the merits of CAPTCHAs.

Cheers,
-- 
Cristóbal Palmer
ibiblio.org
metalab.unc.edu