[Mailman-Users] GSOC idea: mail server/DNS server/mailing list healthcheck

[tlhackque]

On 17-Apr-18 10:28, Rich Kulawiec wrote:
> The idea for this comes from some of the web sites that perform this;
> unfortunately most of them are "upgrading" from simple, fast, easy
> checks to bloated ones that use a ton of Javascript, can't be scripted,
> and are increasingly behind signups/paywalls/etc.
>
> The concept is simple: given a domain, check its DNS, mail, etc.
> setup for correctness and consistency.  For example:
>
> 	- does it have an A record?
> 	- is that valid hostname?
> 	- does it have an AAAA record?
> 	- is that valid hostname?
> 	- does it have MX records?
> 	- are the MX records *not* CNAMEs?
> 	- are they valid hostnames?
> 	- do those hostnames resolve to public IP addresses?
> 	- are any of those MX records duplicates?
> 	- are all the MX responding?
> 	- are the MX weights valid?
> 	- do all MX's pass FCrDNS check?
> 	- does it have NS records?
> 	- are they valid hostnames?
> 	- do they have A, AAAA records?
> 	- are they in public IP space?
> 	- are the NS records *not* CNAMEs?
> 	- do all NS pass FCrDNS check?
> 	- are any of those NS records duplicates?
> 	- does the list of NS match the list of authoritative NS?
> 	- do all the NS return the same list of NS?
> 	- do all the NS return the same list of MX?
> 	- do the NS *not* allow recursion?
> 	- are any of the NS lame?
> 	- are any of the NS missing?
> 	- are all the NS responding?
> 	- is there a working postmaster address?
> 	- is there a working abuse address?
> 	- is there a working hostmaster address?
> 	- if not is there a working address that matches the one in the SOA?
> 	- is the domain's SOA sane?  (e.g. plausible serial number,
> 		refresh, retry, etc.)
> 	- do all the NS return the same SOA with the same serial number?
> 	- is there ASN diversity among the NS?
> 	- and so on
There are plenty of tools for this.  Don't re-invent the wheel.

Here are a few (free) web-based tools (in no particular order); each has
its niche:

http://www.digwebinterface.com/
https://www.ssllabs.com/ssltest/
https://www.huque.com/bin/danecheck
https://dane.sys4.de/
https://mxtoolbox.com/ # many non-mx tests too
https://www.dnsstuff.com/
http://dnscheck.iis.se/
https://www.zonemaster.net/
http://dnsviz.net
https://www.ultratools.com/tools/asnInfo # and many other tools.
http://www.kitterman.com/spf/validate.html
https://www.dmarcanalyzer.com/ # Tools.  All but DMARC Analyzer are
free; it's freemium

Yes, there is a trend toward heavy UIs, but these are not frequent
tests.  If you set things up properly, it's not set and forget - but set
& validate changes need not be frequent enough to care.

If you maintain a few domains, it's not worth scripting.  If you
maintain a lot, the key is picking reputable providers for whatever you
don't do yourself.  If you have decent providers, there's not much for
you to monitor.  If you don't, you have bigger problems.

curl will let you script access to most, or you can use libraries such
as Perl's LWP, WWW::Mechanize::Firefox, WWW::Scripter, WWW::Selenium -
the last few support Javascript by running a browser.

It's better to maintain an interface to tests that someone else
maintains than to try to create & maintain all the tests - many seem
easy, but you'll find it becomes a serious time sink.  In the long term,
there is no such thing as a "small" P.roject.

> Those are the sort of checks that pertain to the operation of any domain
> and its nameservers and mail exchangers.  But in addition, if run on a
> Mailman 2 or 3 host:
>
> 	- what mailing lists are public?
> 	- what mailing lists are private?
> 	- does every list have a working -request address?
> 	- does every list have a working -owner address?
> 	- does every list have a working -bounces address?
> 	- if any list supports the optional -subscribe address,
> 		does it have a -unsubscribe address?
> 	- if any list supports the optional -join address,
> 		does it have a -leave address?
> 	- and so on
These are more interesting for mailman-users.  V3 has an API - but it
can be quite painful to navigate; I opened quite a few issues before
concluding that V3 isn't ready for (my) primetime.

V2 has some CLI tools in /usr/lib/mailman/bin - parsing output can get
you a lot of information.

In either case, it's not particularly interesting what Mailman THINKS is
configured; more things go wrong with bitrot in mailserver configuration
than in Mailman.  So you need to script something that actually tests
each action, receives list mail.  And while you (and I) may use the
CLIs, most users will use a GUI.  You have to test what the customer
uses.  So again, curl and company are your friends.

> Note: command-line tool.  It has to be, because then it can be scripted
> and run via cron and so on.  Besides, anyone running name servers,
> mail servers, etc., had better be able to work at the command line.
>
> Note: should work on systems that aren't running Mailman: just can't
> analyze Mailman then, of course.  This leaves open the door for people
> using other MLMs to write checks that work with those.  And maybe that'd
> be a nice thing to do.
>
> Note: should have varying levels of verbosity, including one that
> explains why something is wrong by referencing RFCs/BCPs/manual by
> chapter and verse.
>
> Note: the second set of checks (above) are outside the scope of what
> Mailman checks inside itself.  That is, they require correlating
> what Mailman thinks should be in place versus what's actually in
> place in the MTA.
>
> ---rsk
>

There's a lot of work to turn this wish list into reality and get it right.

Have fun.  And remember: "A small matter of code usually isn't."

P.S. In your spare time, do start that bird-watching site in Australia. 
I'll be interested to hear how long blocking the US lasts.

-- 
This communication may not represent my employer's views,
if any, on the matters discussed.