[pydotorg-www] Wiki slowness

[Guido van Rossum]

One thing I can say is that some adversaries are very good at coordinating
an attack from a huge pool of IP addresses (e.g. a botnet), at a low rate
per IP address. This type of attack is hard to detect. Presumably each bot
in the net is used to attack many different services simultaneously.

On Wed, Jan 7, 2015 at 1:24 PM, M.-A. Lemburg <mal at egenix.com> wrote:

> On 07.01.2015 21:51, Guido van Rossum wrote:
> > This sounds like a serious and lasting infrastructure issue. Dropbox is
> hit
> > by similar attacks all the time. The hackers are likely looking to match
> > large databases of email addresses (think many, many millions) against
> > large databases of easy passwords; if they find a match they are likely
> to
> > test the same email/password combination at a large set of other
> services.
> > (I can't explain the account creations but these are likely to the
> hackers
> > useful in some other way.)
> >
> > Sooner or later this will be used to hack or impersonate someone
> important.
> >
> > There is no perfect solution, but we should definitely be watching this
> > more carefully and slow down login attempts and account creations. Do we
> > have a captcha yet? Can we block IP addresses? Nothing stops all
> attempts,
> > but you must at least do all of these.
>
> We have a textcha on the account creation page, which blocks
> bots (but not necessarily humans).
>
> The attempts do seem to be programmed, since we're not getting
> a lot of hits for the password reminder link which is on the
> login page as well.
>
> I guess we could try to use fail2ban on the VM which some special
> rules setup to watch for excessive login and account creation
> requests. However, the IP addresses don't repeat often, so
> this may not be all that effective.
>
> MoinMoin itself also has a built-in surge protection:
>
> http://moinmo.in/HelpOnConfiguration/SurgeProtection
>
> but this will likely not help much due to the same problem
> with the varying IP addresses. It also sometimes causes problems
> for people behind firewalls - as we experienced at PyCon UK
> a couple of years ago.
>
> > The times of spambayes are over. The adversaries are persistent and
> clever
> > and have huge resources.
> >
> > (Sadly I can't say much more except over beer. But this is serious.)
>
> The wiki VM runs behind a load balancer, so perhaps we ought
> to look for a more generic solution to install there.
>
> > On Wed, Jan 7, 2015 at 12:36 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> >
> >> I've had a look around on the system at what might be causing the
> >> slowness of the wiki.
> >>
> >> The number of used inodes was a bit high, so I ran some
> >> maintenance tools on the wikis to reduce them.
> >>
> >> A restart of Apache didn't help much. The processes went straight
> >> to 100% again.
> >>
> >> I then ran a log trace of the access log and found that the
> >> wiki us being hit by massive and continuous stream of login attempts
> >> and new account creations. I guess the spammers have us on the
> >> radar again...
> >>
> >> The IP addresses vary a lot, but the user agent strings are mostly
> >> the same: "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64;
> >>  Trident/5.0)".
> >>
> >> All three wikis are affected, so this may be a botnet.
> >>
> >> --
> >> Marc-Andre Lemburg
> >> eGenix.com
> >>
> >> Professional Python Services directly from the Source  (#1, Jan 07 2015)
> >>>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> >> ________________________________________________________________________
> >>
> >> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> >>
> >>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> >>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> >>            Registered at Amtsgericht Duesseldorf: HRB 46611
> >>                http://www.egenix.com/company/contact/
> >> _______________________________________________
> >> pydotorg-www mailing list
> >> pydotorg-www at python.org
> >> https://mail.python.org/mailman/listinfo/pydotorg-www
> >>
> >
> >
> >
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 07 2015)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pydotorg-www/attachments/20150107/66b0da58/attachment-0001.html>