From msapiro at value.net Tue Aug 1 03:12:30 2006 From: msapiro at value.net (Mark Sapiro) Date: Mon, 31 Jul 2006 18:12:30 -0700 Subject: [Mailman-Developers] suggested improvement for Mailman's bounceprocessing In-Reply-To: <20060731115353.M79964@nleaudio.com> Message-ID: Bob Puff wrote: >Whis may be slightly OT, but I just encountered a major problem with the >bounce processing on MM 2.16. Had a large list used for monthly >announcements. I realized that since messages only went out once per month, I >needed to tighten up the default bounce scoring system, so I set it to remove >the user after 1 bounce (score of 1.0). The following day, I saw excessive >CPU load on the box, so I checked into it. It was unsubscribing everyone from >the list, saying due to a bounce in 2005, they were being unsubscribed! I guess this is really a bug. It is cron/disabled doing the unsubscribing. Apparently, back in 2005, everyone bounced once and got a score of 1.0, probably due to some MTA issue. This bounce info is now stale, but stale info is not discarded until a subsequent bounce is received - none were received, so no info was reset, or if a bounce was received more recently, the old info was discarded and replaced with new info, still with score = 1.0. So everyone on the list except recent members who never bounced has a score of 1.0. Then you set the threshold to 1.0 and triggered the code in cron/disabled which is there for just this case. The comment in the code is # Find all the members who are currently bouncing and see if # they've reached the disable threshold but haven't yet been # disabled. This is a sweep through the membership catching # situations where they've bounced a bunch, then the list admin # lowered the threshold, but we haven't (yet) seen more bounces # from the member. Note: we won't worry about stale information # or anything else since the normal bounce processing code will # handle that. In your case, the last sentence is the gotcha. It looks like we need to fix that. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Tue Aug 1 05:21:00 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 31 Jul 2006 23:21:00 -0400 Subject: [Mailman-Developers] Bounces, Pickles, Coding Styles Oh My In-Reply-To: <44C5090B.2080509@mindlace.net> References: <44C5090B.2080509@mindlace.net> Message-ID: <5DB71E00-A1F3-4236-B19F-8B9D67C46156@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jul 24, 2006, at 1:53 PM, emf wrote: > After some non-trivial meditation on BounceRunner.py I think I know > the > difference between verp_probe and verp_bounce, despite not quite > knowing > what VERP means, the comment string for the two functions being > identical, and the bodies of them being identical as well save a tiny > little memory stanza. VERP == Variable Envelop Return Path, although technically we're not actually doing VERP because that has to be done in the MTA. We're doing the closest thing to VERP that an MLM can do. The two functions are a bit different in other ways though, such as the regexp that they use and the fact that verp_probe() pulls data out of the pending database. > Would someone be willing to change verp_probe's docstring to more > clearly illustrate its function? Sure, if I can remember :). Seriously, verp_bounce() parses bounce messages. Probes are similar but a bit different. If we get a bunch of bounces from an address, we'll probe it for a while by sending it messages and seeing if the probes bounce as well. We keep track of the probes in the pending database so that after the required number of probe bounces, we know the address is kaput. > Also, I'm finding a fair number of files that aren't pep8/ > styleguide.txt > happy. anyone care if I at least correct tab/space mixing, and/or make > it otherwise style-guide correct? Are there still files with tabs in them? There definition shouldn't be. Everything should be 4-space indents. As for pep 8 style, I would eventually like to rename things, but as usual, that's more difficult than it sounds because for one thing, it's much harder to port fixes between the trunk and the branch after the rename. We also don't want to do renames (or whitespace normalization for that matter) mixed in with significant changes because the latter will get buried in the diffs. So I've taken the approach that all new code should be pep 8 compliant (with BAW additions :), but that existing stuff shouldn't be renamed... yet. > As much as I appreciate the principles of code-hiding, it's been a bit > of a pain to use the interpreter to figure out what is stored in a > pickle. Heh. And a database is better? :) > Since I can't read the pickle file with my eyes, perhaps a little less > obscuration might be OK? > > Like, > > list.members['i at mindlace.net'] doesn't work. In fact, I can't find any > function that gives me a member in its entirety. That's because there isn't any. Much of the functionality embodied in the various member dicts have been added over time, so usually when we add something, like real names or bounce info, we've added a new dict keyed off the address, rather than worrying about backward compatibility with old dicts. > It would help me if I could just see the underlying data; any > suggestions on how? Yep. Just pop below the MemberAdaptor interface and access the dicts directly. See OldStyleMemberships.py for which dicts contain which data (mostly organized this way for historical reasons). You'll want to use bin/withlist listname to get the pickles loaded first, then explore the MailList attributes from there. Hope that helps. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRM7InHEjvBPtnXfVAQLdtAP/X7c59AVcqJXm033RDqK8v7QfJlfwIxv9 DRXkPCz5LCNtXP/v1UBj9QyUcmoYB9Ze+AkWLQNtWNgyBmtRNvrNBzQPCPKukUNB 0J5a0bNHS6XRK1/dzbVK/yjkwP7EWnBUprIDUNzCBfBn7RFL/4I8EhhsdAGgIIiM 4MlwY7Vmlh0= =pCZh -----END PGP SIGNATURE----- From msapiro at value.net Tue Aug 1 06:31:03 2006 From: msapiro at value.net (Mark Sapiro) Date: Mon, 31 Jul 2006 21:31:03 -0700 Subject: [Mailman-Developers] Bounces, Pickles, Coding Styles Oh My In-Reply-To: <5DB71E00-A1F3-4236-B19F-8B9D67C46156@python.org> Message-ID: Barry Warsaw wrote: > >On Jul 24, 2006, at 1:53 PM, emf wrote: > >> It would help me if I could just see the underlying data; any >> suggestions on how? > >Yep. Just pop below the MemberAdaptor interface and access the dicts >directly. See OldStyleMemberships.py for which dicts contain which >data (mostly organized this way for historical reasons). You'll want >to use bin/withlist listname to get the pickles loaded first, then >explore the MailList attributes from there. Also, point bin/dumpdb at a config.pck to get a view of what's there. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at python.org Tue Aug 1 06:39:05 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 1 Aug 2006 00:39:05 -0400 Subject: [Mailman-Developers] Bounces, Pickles, Coding Styles Oh My In-Reply-To: References: Message-ID: <9ED488CF-4492-48F4-8C81-99E42815B698@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 1, 2006, at 12:31 AM, Mark Sapiro wrote: > Also, point bin/dumpdb at a config.pck to get a view of what's there. Good point, thanks! - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRM7a6nEjvBPtnXfVAQLrhAP/ePjyTFtR5ZlYtyzl8XdBS2Ftaee+lgy4 aRucKKcCObjcanwDW2OTZbXsLB+T8ClDpSeqkyT8rPNm0bMXm2AWbVrCdBqMO3xg UT7nZELBQmfdiax1mJnKp1HNfLZ1VhHTJmLA/ldjxnN6QfUyXPn1qCoOouRWFGkG BPtl1hHLmBw= =v5lb -----END PGP SIGNATURE----- From dandrews at visi.com Tue Aug 1 21:01:21 2006 From: dandrews at visi.com (David Andrews) Date: Tue, 01 Aug 2006 19:01:21 -0000 Subject: [Mailman-Developers] The Philosophy of Web Use. In-Reply-To: References: <084E5E2F4E3B2B637E36FBEE@lcarslon.d.umn.edu> <44AC39EB.6020905@mindlace.net> <33390.72.177.126.107.1152207509.squirrel@mail.his.com> <44AD77B4.8090707@mindlace.net> <44ADCF8A.7080100@mindlace.net> <37460.72.177.126.107.1152245334.squirrel@mail.his.com> Message-ID: <7.0.1.0.2.20060707085000.02bd83e8@visi.com> >On Jul 7, 2006, at 12:08 AM, Brad Knowles wrote: > > >OTOH, I've used Linux and OSX, and before that NeXT, Solaris and >various Unixes for (unfortunately, way :) longer than there's been a >web, and except for the Windows programming I do at work, haven't >ever used IE for any substantial amount of time. I'm not excusing >poor sites or Windows-specific sites, but for the most part, these >days most sites are at least usable with Firefox on various >platforms. (The one sole recent exception was the SQLAlchemy doc >pages which gave Firefox fits but rendered just fine in Safari -- and >I'm sure those guys didn't tailor their pages for IE.). Yeah, you >hit the occasional WMV or ActiveX laden site, but I'm much more >bugged by the Flash-only sites that are an avoidable annoyance for >me, but I can imagine are a scream-out-loud frustration for screen >reader based users. As a screen reader person I will say that Flash-only can be made to work, there is accessible flash, but it is rarely done. If it isn't implemented, a site can be useless, however you can do a good job, my department did a whole web training thing in accessible flash, and it works fine for blind users -- which is good since > > Now, I know that you're not that kind of person, and you will actively > > test your work with MacOS X/Safari, and as many other browser/OS > > platforms > > you can. But the more complexity that is built into the user > > interface, > > the higher the likelihood is that something will accidentally happen > > somewhere to seriously break something for someone else. > >Actually, I think skilled and judicious use of modern web technology >can help to /reduce/ the complexity of Mailman's interface. >Something I constantly struggle with is the plethora of configuration >variables (both via the web u/i and in mm_cfg.py) that makes the >system highly complicated. I would love to have a self-discoverable >interface, or an interface that can be used to selectively reveal >just the parts you're interested. > >- -Barry > >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.4.3 (Darwin) > >iQCVAwUBRK3kGXEjvBPtnXfVAQJfrQQAoYkRLIATvJrLsM2kx88DF4UZHbeoXXZI >CrIhzM2URy+IIcRMQCBn8jfqnyIeTvVUcemHiCjpDyhyVEuYlYXtvjT9d5W9DrjR >OW3rlDViFfuEA6NSU+/QS8YDhokpf2tLpbdKiKLdqgiAdMGR8+jhHFEsmbsha79d >xX7xKMzNmOY= >=/Mp9 >-----END PGP SIGNATURE----- >_______________________________________________ >Mailman-Developers mailing list >Mailman-Developers at python.org >http://mail.python.org/mailman/listinfo/mailman-developers >Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py >Searchable Archives: >http://www.mail-archive.com/mailman-developers%40python.org/ >Unsubscribe: >http://mail.python.org/mailman/options/mailman-developers/dandrews%40visi.com > >Security Policy: >http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp David Andrews and white cane Harry. From a.nielson at griffith.edu.au Wed Aug 2 07:41:08 2006 From: a.nielson at griffith.edu.au (Andrew Nielson) Date: Wed, 2 Aug 2006 15:41:08 +1000 Subject: [Mailman-Developers] HTML Headers Message-ID: Hi, I have written some code to allow Mailman to add a header.html file (if it exists) to the top of each web page. Basically it checks a new option called WEB_HEADER_FILE which is set as follows in mm_cfg.py: WEB_HEADER_FILE = os.path.join(PREFIX, 'Mailman/header.html') I just wanted to know: 1. Would you like me to submit this patch? 2. Is $prefix/Mailman/header.html the correct place for a html file or should we add a html directory or put it in the Cgi one? 3. Would people like me to move the footer stuff out into another footer file and do the same for it? Thanks, Andrew _____________________________________________________ Andrew Nielson Senior Systems Administrator Server Management Services Griffith University Email: A.Nielson at griffith.edu.au Web: http://www.griffith.edu.au/ppages/A_Nielson/ _____________________________________________________ From msapiro at value.net Thu Aug 3 17:25:36 2006 From: msapiro at value.net (Mark Sapiro) Date: Thu, 3 Aug 2006 08:25:36 -0700 Subject: [Mailman-Developers] HTML Headers In-Reply-To: Message-ID: Andrew Nielson wrote: > >I have written some code to allow Mailman to add a header.html file (if it >exists) to the top of each web page. > >Basically it checks a new option called WEB_HEADER_FILE which is set as >follows in mm_cfg.py: >WEB_HEADER_FILE = os.path.join(PREFIX, 'Mailman/header.html') > >I just wanted to know: >1. Would you like me to submit this patch? There is a Google Summer of Code intern working on major modification to the GUI, so your patch may well not be valid and/or relevant soon. You may wish to wait. If you wish to submit it anyway, you can do so at >2. Is $prefix/Mailman/header.html the correct place for a html file or >should we add a html directory or put it in the Cgi one? I think it should be a template, defaulting to an empty file or perhaps a file with an HTML comment documenting what it's for, and accessed via Utils.maketext(). How else would you address i18n issues in a multilingual site? >3. Would people like me to move the footer stuff out into another footer >file and do the same for it? No opinion. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From carbonnb at gmail.com Thu Aug 3 17:45:33 2006 From: carbonnb at gmail.com (Bryan Carbonnell) Date: Thu, 3 Aug 2006 11:45:33 -0400 Subject: [Mailman-Developers] HTML Headers In-Reply-To: References: Message-ID: On 8/2/06, Andrew Nielson wrote: > Hi, > > I have written some code to allow Mailman to add a header.html file (if it > exists) to the top of each web page. > > Basically it checks a new option called WEB_HEADER_FILE which is set as > follows in mm_cfg.py: > WEB_HEADER_FILE = os.path.join(PREFIX, 'Mailman/header.html') > > I just wanted to know: > 1. Would you like me to submit this patch? > 2. Is $prefix/Mailman/header.html the correct place for a html file or > should we add a html directory or put it in the Cgi one? > 3. Would people like me to move the footer stuff out into another footer > file and do the same for it? Just to let you know, I created a patch for 2.1.7 (and 2.1.8 but I apparently haven't uploaded it to Sourceforge yet, oops) that does both sitewide headers and footer, not to mention makes the HTML code XHTML 1.1 compliant. It's patch #1415956 at Sourceforge. -- Bryan Carbonnell - carbonnb at gmail.com Life's journey is not to arrive at the grave safely in a well preserved body, but rather to skid in sideways, totally worn out, shouting "What a great ride!" From lcarlson at d.umn.edu Thu Aug 3 18:31:09 2006 From: lcarlson at d.umn.edu (Laura Carlson) Date: Thu, 03 Aug 2006 11:31:09 -0500 Subject: [Mailman-Developers] The Philosophy of Web Use. In-Reply-To: References: Message-ID: > Ethan wrote: >> Can you do me a favor and let me know how these examples work >> for you? >> >> http://tool-man.org/examples/sorting.html Laura wrote: > Works great for able bodied mouse users. > > But how are people with mobility impairments like low dexterity > (unable to use a pointing device like a mouse and instead must use > keyboard or switch) able to use it? It doesn't seem to work via > keyboard. Hi Ethan, You might want to check out James Edwards' (aka brothercake) latest version of Docking boxes (dbx). It is a more keyboard accessible than the tool-man example. James is making progress in this area. To quote from the site [1]: "Docking boxes (dbx) adds animated drag 'n' drop, snap-to-grid, and show/hide-contents functionality to any group of elements. And ... in what might be another world-first for brothercake - dbx is fully accessible to the keyboard as well as the mouse..." He says in the accessibility notes [2]: Docking boxes can provide equal functionality to both mouse and keyboard users, so the accessibility limitations usually associated with drag 'n' drop controls don't apply. However the dynamic interface is not supported in browser-based screenreaders, such as JAWS or Home Page Reader. Many javascript events occur in browser-based readers exactly as they would for vanilla installs of the host browser (usually Internet Explorer). But having content which is switchable to non-displayed could be a major cause of confusion, in the absence of a reliable way to notify a non-graphical UA of the change explicitly. To try to avoid these problems, the script uses event-disparity evaluations or repositiong routines at all; for these users the layout should remain static, and accessible as though there were no dynamic behaviors, just as it is for legacy or text-only browsers. This then means that CSS properties used to hide the inner content area - things like display:none which would normally be a total no-go - should not be a problem here, since readers should never activate nor cause to be activated the mechanisms which apply those rules. So, in these and other environments where the script is not supported, you'll get a default HTML and CSS layout with no dynamic behaviors. This isn't necessarily an issue for things like navigation or news boxes, where the scripting is supplementary to the core functionality. But its use as an application interface should be restricted to situations where browser and scripting support is predictable, or where equivalent server-side functionality is also provided. That last sentence is key. Restrict use of this technique to situations where: 1. Browser and scripting support is known (it's not for most Mailman admins) or 2. Provide equivalent server-side functionality Best Regards, Laura [1] http://www.brothercake.com/site/resources/scripts/dbx/ [2] http://www.brothercake.com/site/resources/scripts/dbx/setup4/ ___________________________________________ Laura L. Carlson Information Technology Systems and Services University of Minnesota Duluth Duluth, MN U.S.A. 55812-3009 http://www.d.umn.edu/goto/webdesign/ From lcarlson at d.umn.edu Thu Aug 3 18:47:06 2006 From: lcarlson at d.umn.edu (Laura Carlson) Date: Thu, 03 Aug 2006 11:47:06 -0500 Subject: [Mailman-Developers] The Philosophy of Web Use. Message-ID: On Jul 7, 2006, at 12:08 AM, Brad Knowles wrote: >> I'm much more >> bugged by the Flash-only sites that are an avoidable annoyance for >> me, but I can imagine are a scream-out-loud frustration for screen >> reader based users. On August 2, 2006, at 12:08 AM, David Andrews wrote: > As a screen reader person I will say that Flash-only can be made to > work, there is accessible flash, but it is rarely done. If it isn't > implemented, a site can be useless, however you can do a good job, my > department did a whole web training thing in accessible flash, and it > works fine for blind users -- which is good Yes, it's possible to build accessible Flash [1]. In theory, a Flash site should be as accessible as an HTML site; if it provides all the accessibility features that HTML provides, such as alt attributes, keyboard access, synchronized captions for any audio that conveys content, allowing the content to scale to a larger sizes, valid code, links, separating presentation from content, etc. Some flash content even has the potential to aid people with learning or cognitive disabilities. And yes, the more modern versions of screen readers can render it. However, those screen readers can be very expensive, and many users' financial situations preclude even upgrading. There are still a great many people using legacy screen readers out there. In time most people who use screen readers will have software that plays nice with Flash. Until then, though, my advise continues to be to avoid Flash or, if it must be used, to make sure a user has a chance to opt out of it for an accessible page before the Flash begins. Currently the only way to be 100% forward and backward compatible is to create an accessible HTML version of a Flash site. Or only use it for nonessential parts of a web site. Also Flash makes it harder for a site to rank well on the search engines. As far as I know, there are no search engine bots/spiders which can read the contents of Flash. So if you are concerned with SEO you need to include alternative content (ie. text, html) for any Flash content.[2] Best Regards, Laura [1] http://webaim.org/techniques/flash/ [2] http://www.alistapart.com/articles/flashmxmoving/ ___________________________________________ Laura L. Carlson Information Technology Systems and Services University of Minnesota Duluth Duluth, MN U.S.A. 55812-3009 http://www.d.umn.edu/goto/webdesign/ From lcarlson at d.umn.edu Fri Aug 4 13:37:39 2006 From: lcarlson at d.umn.edu (Laura Carlson) Date: Fri, 04 Aug 2006 06:37:39 -0500 Subject: [Mailman-Developers] The Philosophy of Web Use. In-Reply-To: <44D2833B.80402@mindlace.net> References: <44D2833B.80402@mindlace.net> Message-ID: <2B6C6280232BA2AFE3B81054@lcarslon.d.umn.edu> --On Thursday, August 3, 2006 7:14 PM -0400 emf wrote: > You're a goddess, laura. Thanks so much for the docking-boxes hookup. > > ~ethan You're most welcome, Ethan. It may have some potential to aid usability in some circumstances. Just be careful to heed the warnings and restrictions on its use for application interfaces. Laura ___________________________________________ Laura L. Carlson Information Technology Systems and Services University of Minnesota Duluth Duluth, MN U.S.A. 55812-3009 http://www.d.umn.edu/goto/webdesign/ From barry at python.org Fri Aug 4 17:24:50 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 4 Aug 2006 11:24:50 -0400 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7965] branches/Release_2_1-maint/mailman In-Reply-To: References: Message-ID: <4DF9BB80-A933-4B84-A314-A462493DB8ED@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 4, 2006, at 8:21 AM, tkikuchi at users.sourceforge.net wrote: > Revision: 7965 > Author: tkikuchi > Date: 2006-08-04 05:20:33 -0700 (Fri, 04 Aug 2006) > ViewCVS: http://svn.sourceforge.net/mailman/?rev=7965&view=rev > > Log Message: > ----------- > Language files update. New languages: Arabic, Vietnamese. > Hi Tokio! Were you planning on porting these changes to the trunk, or should I? - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRNNmx3EjvBPtnXfVAQLwUQP/b/Ratukf5YVM+6gsLQ43unw4pbSFT2xf itLJO2f5OGRRP1slhp6LftXtBW4dGp9rUwlNBS3fTGQ5ervTX/+oQAVowMaVCJ4S wxohYg84ngrQr2NEjXfZZexTO/iS1xmp5d+iHTLseXezdQByTPzhGY0Wdv4LStyn 4I6yQI4y3aw= =G3Kc -----END PGP SIGNATURE----- From tkikuchi at is.kochi-u.ac.jp Sat Aug 5 04:49:21 2006 From: tkikuchi at is.kochi-u.ac.jp (Tokio Kikuchi) Date: Sat, 05 Aug 2006 11:49:21 +0900 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7965] branches/Release_2_1-maint/mailman In-Reply-To: <4DF9BB80-A933-4B84-A314-A462493DB8ED@python.org> References: <4DF9BB80-A933-4B84-A314-A462493DB8ED@python.org> Message-ID: <44D40731.1040403@is.kochi-u.ac.jp> Hi Barry, Barry Warsaw wrote: > On Aug 4, 2006, at 8:21 AM, tkikuchi at users.sourceforge.net wrote: > >> Revision: 7965 >> Author: tkikuchi >> Date: 2006-08-04 05:20:33 -0700 (Fri, 04 Aug 2006) >> ViewCVS: http://svn.sourceforge.net/mailman/?rev=7965&view=rev >> >> Log Message: >> ----------- >> Language files update. New languages: Arabic, Vietnamese. >> > > Hi Tokio! Were you planning on porting these changes to the trunk, > or should I? > I think I've done it. Shall I start releasing process of 2.1.9 this month? I think I can do it because our university entered in the summer vacation. -- Tokio Kikuchi, tkikuchi at is.kochi-u.ac.jp http://weather.is.kochi-u.ac.jp/ From barry at python.org Mon Aug 7 01:44:46 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 6 Aug 2006 19:44:46 -0400 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7965] branches/Release_2_1-maint/mailman In-Reply-To: <44D40731.1040403@is.kochi-u.ac.jp> References: <4DF9BB80-A933-4B84-A314-A462493DB8ED@python.org> <44D40731.1040403@is.kochi-u.ac.jp> Message-ID: <2A1FA8F3-62E9-48E5-A220-C5E869C0A90B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 4, 2006, at 10:49 PM, Tokio Kikuchi wrote: > I think I've done it. Actually, I don't think so. I just svn up'd on the trunk and I don't see messages/ar or templates/ar, nor do I see the additions in Default.py.in. I do see the Vietnamese entries, but I think Clytie installed those on the trunk a while ago. I'm not sure if the version in the trunk and on the Release_2_1-maint branch are in sync. Could you check again? Thanks, - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iQCVAwUBRNZ+7nEjvBPtnXfVAQLSkAQAlf3TMcbe0p+28TBEvkPLl/rl+m4oIkm7 e0bvoefIXKokL6QG6+o29LEDX67x+Z/CXs65T0OEavmDvQe/M7m7/6MXpaZN6gP5 F0vR2Udyibir3fAp2XRXwDnY1GW3PlDAFOJhaNvyNRXtfNYNvcjQcGNG83lW9vtp YLbNsZwFKgQ= =b3nS -----END PGP SIGNATURE----- From qralston+ml.mailman-developers at andrew.cmu.edu Mon Aug 7 21:08:56 2006 From: qralston+ml.mailman-developers at andrew.cmu.edu (James Ralston) Date: Mon, 07 Aug 2006 15:08:56 -0400 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> Message-ID: <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> On 2006-07-28 at 21:31-05 Brad Knowles wrote: > Unfortunately, there are a whole host of seriously broken MTAs out > there, and seriously broken configurations of otherwise good MTAs, > and many sites return totally bogus status codes. In many cases, > site admins will blindly copy stuff from somewhere else that was > horribly broken to begin with and won't understand what's wrong with > it before they do the cut-n-paste operation. Perhaps, but we cannot solve this problem, and there's a fine line between working around stupidity and coddling it. > That said, I would not be opposed to seeing more data on this > subject, and possibly giving site admins or list admins an option > they can enable that would allow Mailman to pay attention to the > status codes. Once that's out there, we could let various people > try it out and see how it works in the field, and I would be a very > happy guy if I were to be proven wrong in this case. What further data do you wish to see? I think I've documented the problem well enough. There's no way we know many horribly broken sites are out there. On 2006-07-31 at 10:56+01 Ian Eiloart wrote: > I don't see how that could create a problem. The worst thing that > could happen is that someone remains subscribed to a list when they > should not. Alternately, they could be unsubscribed because their > MTA is returning the wrong error codes - but then that would give > their postmaster a good reason to fix the error codes. In this > case, they'd be unsubscribed as things stand anyway. Right: the only risk is that bounces coming from a subscriber at a broken site might be ignored, because they look like they're being generated based on the content of certain messages. IMHO, this risk is negligible. If the operators of the broken site in question get annoyed that Mailman keeps trying to send messages to a non-existent address, they should fix their broken site. As a compromise, I suggest adding this feature as a bounce processing tunable; for example, "content bounce handling": Setting: How should Mailman handle bounces that appear to be related to content? Description: Sometimes a message to a subscriber bounces due to the content of the message, not because the subscriber's address is invalid. This option controls how Mailman handles bounces that appear to be related to the content of messages. Picking "count the bounce" will cause Mailman to count any bounce against the bounce threshold, regardless of the reason why the message bounced. The advantage of this option is that it is least likely to "miss" bounces. The disadvantage of this option is that it penalizes subscribers at sites that correctly indicate why a message bounced. Picking "forward the bounce to the list owner" will cause Mailman to forward bounces that seem to be related to the content of specific messages to the list owner. The advantage of this option is that the owner will be able to review the bounce and take appropriate action; the disadvantage is that the list owner might be overwhelmed by bounces. Picking "ignore the bounce" will cause Mailman to ignore bounces that appear to be related to the content of specific messages. The advantage of this option is that subscribers at sites that correctly indicate why a message bounced won't be penalized. The disadvantage is that if a misconfigured site erroneously indicates that *all* messages are due to content, then Mailman will never detect bouncing subscribers at that site. Choices: [X] Count the bounce against the threshold. [ ] Forward the bounce to the list owner. [ ] Ignore the bounce. Comments? James From qralston+ml.mailman-developers at andrew.cmu.edu Mon Aug 7 21:24:57 2006 From: qralston+ml.mailman-developers at andrew.cmu.edu (James Ralston) Date: Mon, 07 Aug 2006 15:24:57 -0400 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> Message-ID: On 2006-08-07 at 15:08-04 James Ralston wrote: > As a compromise, I suggest adding this feature as a bounce > processing tunable; for example, "content bounce handling": Upon reflection, this problem is yet another instance of Mailman's fundamental problem with bounce processing: Mailman only tracks bounces received per unit of time, not bounces received per messages sent. As an illustration, consider how much easier life would be if bounce processing included an option like this: Disable a subscription if [ ] percent or more of the last [ ] messages bounce. E.g.: Disable a subscription if [20] percent or more of the last [10] messages bounce. Unfortunately, supporting a feature like this would require maintaining [n] separate databases for every mailing list (where [n] is the number of subscribers to the list), and updating all [n] databases every time a message is sent to the list. I'm not sure if doing so would be feasible... James From bob at nleaudio.com Mon Aug 7 21:28:46 2006 From: bob at nleaudio.com (Bob Puff@NLE) Date: Mon, 07 Aug 2006 15:28:46 -0400 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> Message-ID: <44D7946E.60300@nleaudio.com> James Ralston wrote: > Choices: > > [X] Count the bounce against the threshold. > [ ] Forward the bounce to the list owner. > [ ] Ignore the bounce. > > Comments? I thought there already was a "Forward bounces to admin" setting. If not, there should be (and derfault should be off). Then, this question should be, "Try to interpret content bounces? Y/N". It gets messy (and often confusing) combining two selections with one question. IMHO, I really don't care to try to determine content bounces. I've seen many that give no indication of why the message bounced. The bounce processor I wrote for 2.0.x handles these in what I consider an appropriate way: 1. If a user has bounces every day messages are delivered, and this continues for x days, they get canned. 2. If a day goes by that messages are delivered, and no bounce occurs, bounce info is reset. If people bounce a message every day for a couple weeks, I consider their ISP broken enough to warrant unsubscription. Bob P.S. Tokio & Barry: please don't forget to have a look at the bounce issue I posted a week or so regarding - that's a nasty one! From qralston+ml.mailman-developers at andrew.cmu.edu Mon Aug 7 22:26:08 2006 From: qralston+ml.mailman-developers at andrew.cmu.edu (James Ralston) Date: Mon, 07 Aug 2006 16:26:08 -0400 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <44D7946E.60300@nleaudio.com> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> Message-ID: <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> On 2006-08-07 at 15:28-04 Bob Puff at NLE wrote: > James Ralston wrote: > > > Choices: > > > > [X] Count the bounce against the threshold. > > [ ] Forward the bounce to the list owner. > > [ ] Ignore the bounce. > > I thought there already was a "Forward bounces to admin" setting. No, that's another patch I'm working on. ;) But that's not the goal here. The goal of the "forward the bounce" option here is "I want to see what type of stuff would be ignored if I picked 'Ignore the bounce'." In other words, it's a mechanism to permit a list owner to make a determination whether the "Ignore the bounce" setting is appropriate for his specific list, which Brad obliquely suggested. > IMHO, I really don't care to try to determine content bounces. As a list owner, you shouldn't need to care. Mailman should just Do The Right Thing. My argument is that ignoring content-related bounces is the Right Thing. Actually, I can phrase it more strongly: as it currently stands, Mailman's lack of attention to content-related bounces *requires* me to violate RFC2822. If I refuse to do so, our users' subscriptions at remote Mailman sites will be disabled. James From brad at stop.mail-abuse.org Tue Aug 8 03:44:07 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Mon, 7 Aug 2006 20:44:07 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> Message-ID: At 4:26 PM -0400 2006-08-07, James Ralston wrote: > As a list owner, you shouldn't need to care. Mailman should just Do > The Right Thing. My argument is that ignoring content-related bounces > is the Right Thing. The problem is determining, in a programmatic and systematic way, what really is a content-related bounce and what might mistakenly appear to be a content-related bounce, and the converse. Then look at what happens when you make the guess the wrong way, what potential additional "cost" there may be to the system for a false positive versus a false negative, and add some weightings to the situation so as to try to minimize the overall drawbacks to such a technique. The SpamAssassin people do this kind of analysis on a massive amount of spam that they have collected over the years, when re-running their complete collection of rule weightings to try to find an optimum setting. Problem is, it takes them something like a month to make a single complete run through all the rules with all the input spam, to come up with a given set of proposed set of weightings -- and this is on a large set of distributed servers, in a manner somewhat akin to SETI at Home. At that point, they're ready to release a new version, because more rules and techniques have been introduced since the last version they released and the weights have also been updated, and they start the whole process all over again. Now, we're not talking about something quite that intensive, but it could still be a pretty big affair to make sure that we're striking the proper balance of risking false positives versus false negatives. As it stands today, it's just some people talking about abstract theory. No one has collected any appreciable amount of bounce information to tell us what the real-world picture is at their site. If you want to move this discussion beyond the theory stage, I'd suggest that you start collecting some data. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From brad at stop.mail-abuse.org Tue Aug 8 03:35:06 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Mon, 7 Aug 2006 20:35:06 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> Message-ID: At 3:08 PM -0400 2006-08-07, James Ralston wrote: > Perhaps, but we cannot solve this problem, and there's a fine line > between working around stupidity and coddling it. Right, but if we can't fix the problem of the multitude of broken MTAs out there, and the fact that most of them probably don't assign the appropriate extended response codes in accordance with the RFCs, then the likelihood is that we are going to be lead to make the wrong guesses based on the response we get. I think the question is how damaging are those wrong guesses, and as compared to not making any attempt to guess one way or the other and just treat all bounces as the same? Without any further detailed information, my gut feeling is that we're better off not trying to guess what the real reason was for a given bounce, but to just treat them all the same and to give enough lattitude that people don't get unsubscribed with just a single bounce (or whatever). > What further data do you wish to see? I think I've documented the > problem well enough. There's no way we know many horribly broken > sites are out there. Save a copy of each and every bounce you get over an extended period of time (this may require modifications to the source code), and then try to categorize them by the easy-to-parse numeric response code versus the harder-to-parse description, and actually find out how the cookie crumbles. Describing the one particular type of sub-problem that you've run into doesn't really help us in this situation, not when you're talking about changing the behaviour of an entire subsystem in order to accommodate your one specific issue. Instead, you need to go on a quest to obtain large amounts of data that demonstrate how easy (or hard) it is to determine the real reason why some message bounced and then figure out how you can take that information and modify the source code to suit. > Right: the only risk is that bounces coming from a subscriber at a > broken site might be ignored, because they look like they're being > generated based on the content of certain messages. I'm not convinced that's the only risk, and I'm not convinced that the potential consequences are that minor. But if you can provide sufficient evidence to show that you are correct, at least for the users on your site, I'm willing to be convinced. > IMHO, this risk is negligible. If the operators of the broken site in > question get annoyed that Mailman keeps trying to send messages to a > non-existent address, they should fix their broken site. Well, if the windmill turns out to be Microsoft, you might want to seriously think about whether or not you really want to continue trying to tilt at that thing. You might want to look into how big this problem could potentially be, before you decide to just casually blow off any possible consequences. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Tue Aug 8 11:42:54 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 10:42:54 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> Message-ID: --On 7 August 2006 15:08:56 -0400 James Ralston wrote: > > Choices: > > [X] Count the bounce against the threshold. > [ ] Forward the bounce to the list owner. > [ ] Ignore the bounce. > > Comments? > The default should NOT be to count the bounce against the threshold. Let's not assume that all remote servers are broken. I'd guess that the default should be to ignore the bounce. Another option might be to bounce to the original sender. In fact, that could be a *per user* option, though that would require more work. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 11:46:48 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 10:46:48 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <44D7946E.60300@nleaudio.com> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> Message-ID: <7B6D4B2624B4903E3C2B1B88@lewes.staff.uscs.susx.ac.uk> --On 7 August 2006 15:28:46 -0400 "Bob Puff at NLE" wrote: > > > James Ralston wrote: > >> Choices: >> >> [X] Count the bounce against the threshold. >> [ ] Forward the bounce to the list owner. >> [ ] Ignore the bounce. >> >> Comments? > > I thought there already was a "Forward bounces to admin" setting. If > not, there should be (and derfault should be off). Then, this question > should be, "Try to interpret content bounces? Y/N". > > It gets messy (and often confusing) combining two selections with one > question. > > IMHO, I really don't care to try to determine content bounces. I've seen > many that give no indication of why the message bounced. The bounce > processor I wrote for 2.0.x handles these in what I consider an > appropriate way: > > 1. If a user has bounces every day messages are delivered, and this > continues for x days, they get canned. > 2. If a day goes by that messages are delivered, and no bounce occurs, > bounce info is reset. I've been unsubscribed from Yahoo lists because my server rejects illegally formatted messages. That's good enough reason for me to support distinguishing between recipient errors and content errors. > If people bounce a message every day for a couple weeks, I consider their > ISP broken enough to warrant unsubscription. In my case, it wasn't my ISP, or my server that was at fault. It was the message *senders* mail client that was constructing faulty headers. > Bob > > P.S. Tokio & Barry: please don't forget to have a look at the bounce > issue I posted a week or so regarding - that's a nasty one! > > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.a > c.uk > > Security Policy: > http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 11:55:40 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 10:55:40 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> Message-ID: <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> --On 7 August 2006 20:44:07 -0500 Brad Knowles wrote: > At 4:26 PM -0400 2006-08-07, James Ralston wrote: > >> As a list owner, you shouldn't need to care. Mailman should just Do >> The Right Thing. My argument is that ignoring content-related bounces >> is the Right Thing. > > The problem is determining, in a programmatic and systematic way, > what really is a content-related bounce and what might mistakenly > appear to be a content-related bounce, and the converse. No, that isn't the problem. The RFC says how to do this, and we should trust the RFC. If people have broken servers then actually there's nothing that can go wrong which isn't already going wrong. > ....> > If you want to move this discussion beyond the theory stage, I'd > suggest that you start collecting some data. I can't see that data is required. There are two categories of error, and the consequences are neutral in both cases: 1. A message is labelled as a content bounce when it's really a recipient bounce. The consequence is that the recipient stays subscribed. This isn't a real problem. The worst that happens is a bit of extra traffic, or that the admin reverts to the old behaviour. 2. The message is labelled as a recipient bounce when it's really a content bounce. This is the status quo. People may already be incorrectly unsubscribed. This is a real problem when it occurs. It can happen because a server refuses messages with illegal (RFC non-compliant) headers, as well as when the content is offensive. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 11:56:41 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 10:56:41 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> Message-ID: <2802265948583FA8D00A5D02@lewes.staff.uscs.susx.ac.uk> --On 7 August 2006 20:35:06 -0500 Brad Knowles wrote: > At 3:08 PM -0400 2006-08-07, James Ralston wrote: > >> Perhaps, but we cannot solve this problem, and there's a fine line >> between working around stupidity and coddling it. > > Right, but if we can't fix the problem of the multitude of broken > MTAs out there, and the fact that most of them probably don't assign > the appropriate extended response codes in accordance with the RFCs, > then the likelihood is that we are going to be lead to make the wrong > guesses based on the response we get. We already do that. This is the problem that we're trying to solve, not a new problem introduced by the proposal! -- Ian Eiloart IT Services, University of Sussex From brad at stop.mail-abuse.org Tue Aug 8 12:00:17 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 8 Aug 2006 05:00:17 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <7B6D4B2624B4903E3C2B1B88@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7B6D4B2624B4903E3C2B1B88@lewes.staff.uscs.susx.ac.uk> Message-ID: At 10:46 AM +0100 2006-08-08, Ian Eiloart quoted "Bob Puff at NLE" : >> If people bounce a message every day for a couple weeks, I consider their >> ISP broken enough to warrant unsubscription. > > In my case, it wasn't my ISP, or my server that was at fault. It was the > message *senders* mail client that was constructing faulty headers. Right, but that's Yahoo. That's not Mailman. Mailman is unlikely to be doing this sort of thing. If anything, it would most likely be scrubbing the messages in order to remove illegal formatting. I can understand the overall desire in this specific case, but I'm having a hard time painting Mailman with that same brush, which would then reasonably lead to a requirement to make significant changes to the Mailman bounce handling scheme in order to try and guess as to what was the real reason behind a particular type of bounce. I'm not saying that this isn't something that we shouldn't at least look at seriously, I'm just saying I don't quite buy this particular motivation, at least not as it applies to Mailman. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From brad at stop.mail-abuse.org Tue Aug 8 12:10:41 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 8 Aug 2006 05:10:41 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> Message-ID: At 10:55 AM +0100 2006-08-08, Ian Eiloart wrote: >> The problem is determining, in a programmatic and systematic way, >> what really is a content-related bounce and what might mistakenly >> appear to be a content-related bounce, and the converse. > > No, that isn't the problem. The RFC says how to do this, and we should > trust the RFC. If people have broken servers then actually there's > nothing that can go wrong which isn't already going wrong. And if Yahoo jumps off a bridge because they think the RFC tells them to do that, what should we do? And what if AOL, pobox.com, hotmail.com, and all the other big providers make the same mistake? When it comes to parsing the actual reasons behind a message bouncing, the RFC is not sufficient. Indeed, I'm not convinced that it's even necessary. And you'd have to be specific which RFC you're talking about, because some of them are mutually incompatible. Trust me, this is a more complex subject than you think it is. And just blindly applying what you think is the right solution is likely to cause a lot more pain for you and for everyone else, and not necessarily for any real good purpose when everything is said and done. > I can't see that data is required. Then go ahead and make the change, and then tell us how it works out for you. > There are two categories of error, and > the consequences are neutral in both cases: > > 1. A message is labelled as a content bounce when it's really a recipient > bounce. Or some other kind of bounce. > The consequence is that the recipient stays subscribed. This isn't a > real problem. The worst that happens is a bit of extra traffic, or that > the admin reverts to the old behaviour. This can be a very real problem for admins that are running larger sites, and already handling large amounts of traffic. If the admin is forced to disable some new feature in order to restore his site to a reasonably well working state, then he's not likely to make that upgrade. > 2. The message is labelled as a recipient bounce when it's really a > content bounce. Or some other kind of bounce. > This is the status quo. People may already be incorrectly unsubscribed. > This is a real problem when it occurs. It can happen because a server > refuses messages with illegal (RFC non-compliant) headers, as well as when > the content is offensive. Or when the server looks up your IP address and finds it on a black list, or thinks it finds it on a blacklist which no longer exists, or any number of other problems. We can't fix the entire Internet, and when people have misconfigured their servers to generate inappropriate types of bounces, there's not much we can do to help them. In my experience, any kind of guess that we might be able to make programmatically is usually wrong for a statistically significant subset of the population. Moreover, the potential damage from false positives or false negatives is usually worse for the collective whole than simply not trying to guess one way or the other, and to just give people enough lattitude that it shouldn't matter. But you've got the opportunity here to generate real-world data on how this process works, and to put the whole issue to rest. Please let us know how it works out for you. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From brad at stop.mail-abuse.org Tue Aug 8 12:13:37 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 8 Aug 2006 05:13:37 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <2802265948583FA8D00A5D02@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <2802265948583FA8D00A5D02@lewes.staff.uscs.susx.ac.uk> Message-ID: At 10:56 AM +0100 2006-08-08, Ian Eiloart wrote: >> Right, but if we can't fix the problem of the multitude of broken >> MTAs out there, and the fact that most of them probably don't assign >> the appropriate extended response codes in accordance with the RFCs, >> then the likelihood is that we are going to be lead to make the wrong >> guesses based on the response we get. > > We already do that. This is the problem that we're trying to solve, not a > new problem introduced by the proposal! No, that's precisely the problem -- the proposal does cause new problems that have to be dealt with. Because of all the broken MTAs out there, I believe that the probability is high that we will be unable to guess correctly what type of bounce we have for a statistically significant subsection of the population, and that the potential consequences of either a false negative or a false positive in this case are higher than taking the K.I.S.S. approach and not making any attempt to guess what type of bounce we're dealing with. So, feel free to go ahead and make this change and to put this entire issue to rest, at least for the data you've collected from your site. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Tue Aug 8 12:16:01 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 11:16:01 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7B6D4B2624B4903E3C2B1B88@lewes.staff.uscs.susx.ac.uk> Message-ID: --On 8 August 2006 05:00:17 -0500 Brad Knowles wrote: > At 10:46 AM +0100 2006-08-08, Ian Eiloart quoted "Bob Puff at NLE" > : > >>> If people bounce a message every day for a couple weeks, I consider >>> their ISP broken enough to warrant unsubscription. >> >> In my case, it wasn't my ISP, or my server that was at fault. It was the >> message *senders* mail client that was constructing faulty headers. > > Right, but that's Yahoo. That's not Mailman. Mailman is unlikely to be > doing this sort of thing. If anything, it would most likely be scrubbing > the messages in order to remove illegal formatting. Really, Mailman is fixing up message header syntax? > I can understand the overall desire in this specific case, but I'm having > a hard time painting Mailman with that same brush, which would then > reasonably lead to a requirement to make significant changes to the > Mailman bounce handling scheme in order to try and guess as to what was > the real reason behind a particular type of bounce. > > > I'm not saying that this isn't something that we shouldn't at least look > at seriously, I'm just saying I don't quite buy this particular > motivation, at least not as it applies to Mailman. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 12:27:58 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 11:27:58 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> Message-ID: <13BCE0C17042B61DD32915CC@lewes.staff.uscs.susx.ac.uk> --On 8 August 2006 05:10:41 -0500 Brad Knowles wrote: > > Or some other kind of bounce. > >> This is the status quo. People may already be incorrectly >> unsubscribed. This is a real problem when it occurs. It can happen >> because a server refuses messages with illegal (RFC non-compliant) >> headers, as well as when the content is offensive. > > Or when the server looks up your IP address and finds it on a black > list, or thinks it finds it on a blacklist which no longer exists, or > any number of other problems. It doesn't matter. The point is that they're NOT saying the recipient doesn't exist. Currently, we treat this situation as if they do, and we shouldn't! -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 12:29:08 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 11:29:08 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <2802265948583FA8D00A5D02@lewes.staff.uscs.susx.ac.uk> Message-ID: <201E3200B83A92A981496F9C@lewes.staff.uscs.susx.ac.uk> --On 8 August 2006 05:13:37 -0500 Brad Knowles wrote: > At 10:56 AM +0100 2006-08-08, Ian Eiloart wrote: > >>> Right, but if we can't fix the problem of the multitude of broken >>> MTAs out there, and the fact that most of them probably don't assign >>> the appropriate extended response codes in accordance with the RFCs, >>> then the likelihood is that we are going to be lead to make the wrong >>> guesses based on the response we get. >> >> We already do that. This is the problem that we're trying to solve, not >> a new problem introduced by the proposal! > > No, that's precisely the problem -- the proposal does cause new problems > that have to be dealt with. Well, that's not true if the new default behaviour is the current broken behaviour. Would you accept that? > Because of all the broken MTAs out there, I believe that the probability > is high that we will be unable to guess correctly what type of bounce we > have for a statistically significant subsection of the population, and > that the potential consequences of either a false negative or a false > positive in this case are higher than taking the K.I.S.S. approach and > not making any attempt to guess what type of bounce we're dealing with. > > > So, feel free to go ahead and make this change and to put this entire > issue to rest, at least for the data you've collected from your site. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 12:46:34 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 11:46:34 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> Message-ID: --On 8 August 2006 05:10:41 -0500 Brad Knowles wrote: > > When it comes to parsing the actual reasons behind a message > bouncing, the RFC is not sufficient. Indeed, I'm not convinced that > it's even necessary. And you'd have to be specific which RFC you're > talking about, because some of them are mutually incompatible The original proposal referred to error codes defined in rfc1893. The parsing of such error codes is relatively trivial. Which RFCs are incompatible? Here's a more specific proposal: 1. If there is no rfc1893 error code, then treat the message as a recipient bounce (count it against the recipient address). 2. If there is an rfc 1893 code, and the first digit is '2' or '4' then do nothing. 3. If there is an rfc 1893 code and the first digit is '5', then do nothing, except as noted below, where count+ means count against the recipient, count- means count in favour, admin means optionally notify the administrator (who may wish to notify the remote admin, or may even happen to be be the remote admin). X.1.0 Other address status X.1.1 Bad destination mailbox address count+ X.1.2 Bad destination system address count+ X.1.3 Bad destination mailbox address syntax count+ X.1.4 Destination mailbox address ambiguous count+ X.1.5 Destination mailbox address valid count- X.1.6 Mailbox has moved count+ X.1.7 Bad sender's mailbox address syntax X.1.8 Bad sender's system address X.2.0 Other or undefined mailbox status X.2.1 Mailbox disabled, not accepting messages count+ X.2.2 Mailbox full X.2.3 Message length exceeds administrative limit. X.2.4 Mailing list expansion problem admin X.3.0 Other or undefined mail system status X.3.1 Mail system full admin X.3.2 System not accepting network messages X.3.3 System not capable of selected features X.3.4 Message too big for system X.4.0 Other or undefined network or routing status X.4.1 No answer from host X.4.2 Bad connection X.4.3 Routing server failure X.4.4 Unable to route X.4.5 Network congestion X.4.6 Routing loop detected X.4.7 Delivery time expired X.5.0 Other or undefined protocol status X.5.1 Invalid command X.5.2 Syntax error X.5.3 Too many recipients X.5.4 Invalid command arguments X.5.5 Wrong protocol version X.6.0 Other or undefined media error X.6.1 Media not supported X.6.2 Conversion required and prohibited X.6.3 Conversion required but not supported X.6.4 Conversion with loss performed X.6.5 Conversion failed Actually, for the moment, I'd be happy if everything except count+ were ignored (but logged). In future, it may be possible to implement other desirable behaviours for other error codes. In particular, the local sysadmin may want to know about many of these things when the recipient is local. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Tue Aug 8 13:20:29 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 12:20:29 +0100 Subject: [Mailman-Developers] Please confirm your message In-Reply-To: <1155030421.24086.TMDA@web3.nlenet.net> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7B6D4B2624B4903E3C2B1B88@lewes.staff.uscs.susx.ac.uk> <1155030421.24086.TMDA@web3.nlenet.net> Message-ID: <72A1B6B13AC093EDAD65ABB6@lewes.staff.uscs.susx.ac.uk> Bob, This is very annoying. --On 8 August 2006 05:47:01 -0400 Bob Puff wrote: > Hello iane at sussex.ac.uk, > > ** IMPORTANT! Please Read! ** > In an effort to reduce spam, a message filtering service called TMDA has > been implemented on this mailbox. > > The purpose of this message is to verify that you, a live human, did > indeed sent a message (shown below) to this mailbox. > > It is IMPORTANT that you do one of the following two things: > > > 1. Simply reply to THIS message (you don't need to type anything, just > send a blank message), or > > 2. Click on this link: > http://nleaudio.com/cgi-bin/tmda.cgi?501.1155030421.24086.12f493 > > > After doing so, your messages will go through automatically, and you will > not see this message again. If you do not reply to this email within > three weeks, your original message may be lost. > > Thank you for your patience. This one small step saves this user from a > ton of spam. > -- Ian Eiloart IT Services, University of Sussex From bob at nleaudio.com Tue Aug 8 17:14:22 2006 From: bob at nleaudio.com (Bob Puff) Date: Tue, 8 Aug 2006 11:14:22 -0400 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> Message-ID: <20060808151106.M99326@nleaudio.com> I've deliberately not quoted the original message, but Brad is 100% on the money in his posts. I see a bunch of bounces that one of my highly-customized mailman lists get. At one point, I tried keeping up with just the parsing of the bounce messages, but soon gave up. There are too many strange ones, and ones that give you no clue for the reason of the bounce... some give you no clue WHO bounced! I've even seen some that come back from different domains. Like Brad said, the KISS rule really is a good idea. Bob From lcarlson at d.umn.edu Tue Aug 8 17:15:18 2006 From: lcarlson at d.umn.edu (Laura Carlson) Date: Tue, 08 Aug 2006 10:15:18 -0500 Subject: [Mailman-Developers] Drag and Drop Articles In-Reply-To: <2B6C6280232BA2AFE3B81054@lcarslon.d.umn.edu> References: <44D2833B.80402@mindlace.net> <2B6C6280232BA2AFE3B81054@lcarslon.d.umn.edu> Message-ID: <878AAA0FE117EA5703C994CD@lcarslon.d.umn.edu> --On Thursday, August 3, 2006 7:14 PM -0400 emf wrote: > Thanks so much for the docking-boxes hookup. > > ~ethan fyi...some drag and drop usability articles: Drag and Drop Controls By Free Usability Advice. "Question: Are there any usability issues with using drag and drop controls in a web application?" Answer: "AJAX technology makes it easier to add drag and drop controls to a web application...Since web technologies haven't commonly supported drag and drop until now, it's difficult for users to understand when an object in the UI is draggable or droppable. It is especially difficult since the objects don't usually look different from objects that do not have this functionality. Adding visual affordances such as shading, cursor changes, text and other elements can help to communicate that an item is draggable...Before you decide to implement drag and drop, consider whether it is the right solution for your users. Don't just implement new technology for new technology's sake. If you do decide to use drag and drop functionality, make sure to give users a way to notice and learn the interaction." http://freeusabilityadvice.com/archive/28/drag-and-drop-controls Usability by Hand, AJAX and Efficiency By Thomas Baekdal. "We have seen a huge amount of drag and drop...Some work, some are very interesting, but most of them directly kill your efficiency...Take drag and drop shopping carts...drag and drop is over 1000% slower than clicking an 'add to cart' button." http://www.baekdal.com/articles/Usability/usability-ajax-efficiency/ Drag 'n Drop is Invisible To Users By Jared Spool. "...They [Netflix] offers a margin feature that allows their users to see the top items in their movie queue. And, to give users that extra umph of interface goodness, you can reorder the queue with drag-and-drop. What Netflix figured out was that nobody else would figure it out unless they included instructions. So, that's exactly what they do. In a very concise box at the bottom of the list, they include a Helpful Tip that visually demonstrates the drag-and-drop capability." http://tinyurl.com/h38kx Laura ___________________________________________ Laura L. Carlson Information Technology Systems and Services University of Minnesota Duluth Duluth, MN U.S.A. 55812-3009 http://www.d.umn.edu/goto/webdesign/ From iane at sussex.ac.uk Tue Aug 8 17:44:19 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 08 Aug 2006 16:44:19 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <20060808151106.M99326@nleaudio.com> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> Message-ID: <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> --On 8 August 2006 11:14:22 -0400 Bob Puff wrote: > > I've deliberately not quoted the original message, but Brad is 100% on the > money in his posts. I see a bunch of bounces that one of my > highly-customized mailman lists get. At one point, I tried keeping up > with just the parsing of the bounce messages, but soon gave up. There > are too many strange ones, and ones that give you no clue for the reason > of the bounce... some give you no clue WHO bounced! I've even seen some > that come back from different domains. > > Like Brad said, the KISS rule really is a good idea. > > Bob But, the idea is NOT to try to parse bounce *messages*, it's to parse bounce *codes*. -- Ian Eiloart IT Services, University of Sussex From brad at stop.mail-abuse.org Tue Aug 8 19:16:35 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 8 Aug 2006 12:16:35 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> Message-ID: At 4:44 PM +0100 2006-08-08, Ian Eiloart wrote: > But, the idea is NOT to try to parse bounce *messages*, it's to parse > bounce *codes*. Here's the deal. You think it's going to be trivially easy to add this new feature, and to parse the codes correctly, with the correct outcome, all you have to do is follow the RFC and everything will be hunky-dory. I think that the problem is a lot more complex than that, with many sites giving totally inappapropriate response codes for the real underlying reason, and trying to parse them is likely to cause more problems than it solves. Moreover, I think this is going to add unneeded complexity to the system for what I believe will be, at best, relatively minimal benefit. Worse, in order to adhere to the spirit of this idea and make the concept actually work, we'll have to get into trying to parse the actual wording of the error messages, and then we'll have to get into internationalization issues of all those words we're trying to parse, because I'm pretty sure that words like "virus" are not the same in Polish, Chinese, Farsi, and whatever other various languages we have to support. So, here's the solution. You go implement the code to do what you're talking about, and see how it works on your site. Make sure to collect all the bounce messages in question, and the action that was taken by the system. This way, we humans can compare the performance of your new code. Once you're done tweaking the system to work as well as you can manage, come back to us and show us your code and your input data, and prove to us how well it works. But without a patch and a strong indication that this is a significant improvement for relatively little added complexity, I don't think you're going to get any further traction in this issue. That's it. I've said my piece. Unless you have something new to add to the discussion, I'd suggest you do us all a favour and let this drop. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Wed Aug 9 11:49:24 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Wed, 09 Aug 2006 10:49:24 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> Message-ID: <84389B4CD9311B2303DF716C@lewes.staff.uscs.susx.ac.uk> --On 8 August 2006 12:16:35 -0500 Brad Knowles wrote: > I think that the problem is a lot more complex than that, with many sites > giving totally inappapropriate response codes for the real underlying > reason, and trying to parse them is likely to cause more problems than it > solves. So, give us one example of a problem that could arise. -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Wed Aug 9 11:49:57 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Wed, 09 Aug 2006 10:49:57 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> Message-ID: --On 8 August 2006 12:16:35 -0500 Brad Knowles wrote: > Worse, in order to adhere to the spirit of this idea and make the > concept actually work, we'll have to get into trying to parse the actual > wording of the error messages, and then we'll have to get into > internationalization issues of all those words we're trying to parse, > because I'm pretty sure that words like "virus" are not the same in > Polish, Chinese, Farsi, and whatever other various languages we have to > support. Rubbish. The codes are numbers. -- Ian Eiloart IT Services, University of Sussex From brad at stop.mail-abuse.org Wed Aug 9 19:04:17 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Wed, 9 Aug 2006 12:04:17 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <84389B4CD9311B2303DF716C@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> <84389B4CD9311B2303DF716C@lewes.staff.uscs.susx.ac.uk> Message-ID: At 10:49 AM +0100 2006-08-09, Ian Eiloart wrote: > So, give us one example of a problem that could arise. I don't have to. If you want people to believe you, then you need to prove that you can create a significant enhancement to Mailman in this area, without creating a significant increase in the complexity of the system, or at least with a reasonable balance of complexity versus the modified features you're proposing. I don't have anything to prove here. You do. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From brad at stop.mail-abuse.org Wed Aug 9 19:18:13 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Wed, 9 Aug 2006 12:18:13 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> Message-ID: At 10:49 AM +0100 2006-08-09, Ian Eiloart wrote: > Rubbish. The codes are numbers. Right. But, as I said, the codes aren't going to be sufficient. They're going to be misleading in many cases, causing us to make inappropriate conclusions based on faulty information. Therefore, if you want to uphold the spirit of what you're asking for, you're going to have to look deeper and try to start parsing the actual text of the bounce message to try to better understand what the real reason was. And that way lies madness. Otherwise, RFC-1893 would have been sufficient to answer all possible questions about this feature, and all MTA authors and all mail systems administrators would have been able to perfectly follow those guidelines. We wouldn't have needed RFC 3463, or the updates from RFCs 3886, 4468, etc.... The fact that there was some perceived ambiguity lead to confusion and inappropriate implementation, and incompatibility. Which lead to newer RFCs being written on this subject in order to try to clarify the situation and hopefully lead to greater compatibility. Unfortunately, there's still lots of old code and old installations out there, and they are unable or unwilling to upgrade, so now you've got all this legacy code you're saddled with, along with all this new code as well. So, if you build your parser to handle exclusively RFC 4468 codes, and someone has written or implemented an MTA using codes from 1893 that they misinterpreted, you're probably going to have a hard time figuring out what they meant and why. Keep in mind that you're not only fighting MTA authors here, but also the vast majority of clueless MTA administrators that take a recommended configuration from someone else that is likely to be wrong and apply it inappropriately at their site, and thus perpetuate and worsen the problem far beyond the level of damage that MTA authors would ever possibly be capable of -- and MTA authors are capable of screwing up a whole lot of stuff. So, show me a parser that fully understands all possible correct interpretations of these RFCs, plus all possible incorrect but likely interpretations of these RFCs, and we might have something useful to talk about. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Thu Aug 10 12:21:49 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Thu, 10 Aug 2006 11:21:49 +0100 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> Message-ID: <558AE7B6F747797A2A8B79BD@lewes.staff.uscs.susx.ac.uk> --On 9 August 2006 12:18:13 -0500 Brad Knowles wrote: > > Otherwise, RFC-1893 would have been sufficient to answer all possible > questions about this feature, and all MTA authors and all mail systems > administrators would have been able to perfectly follow those guidelines. > We wouldn't have needed RFC 3463, or the updates from RFCs 3886, 4468, > etc.... > Neither RFC makes any significant changes here. "Appendix B - Changes from RFC1893 Changed Authors contact information. Updated required standards boilerplate. Edited the text to make it spell-checker and grammar checker compliant. Modified the text describing the persistent transient failure to more closely reflect current practice and understanding. Eliminated the restriction on the X.4.7 codes limiting them to persistent transient errors." RFC 4468 adds two new codes. > So, show me a parser that fully understands all possible correct > interpretations of these RFCs, plus all possible incorrect but likely > interpretations of these RFCs, and we might have something useful to talk > about. > It's not necessary to understand all interpretations. There are a few codes that mean the remote address isn't available. When we see any other code, we should not count the bounce against the specific address, because the error isn't related to that address. -- Ian Eiloart IT Services, University of Sussex From brad at stop.mail-abuse.org Thu Aug 10 17:01:45 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Thu, 10 Aug 2006 10:01:45 -0500 Subject: [Mailman-Developers] suggested improvement for Mailman's bounce processing In-Reply-To: <558AE7B6F747797A2A8B79BD@lewes.staff.uscs.susx.ac.uk> References: <77D33A972456F01D4CD978F4@emerald.sei.cmu.edu> <4AB65DCD3D5997D142E7B517@emerald.sei.cmu.edu> <44D7946E.60300@nleaudio.com> <7FC33E32714B236AD9A6886E@emerald.sei.cmu.edu> <1E894427A8E7E144B9D2176B@lewes.staff.uscs.susx.ac.uk> <20060808151106.M99326@nleaudio.com> <89EEB8D292650BB95880737C@lewes.staff.uscs.susx.ac.uk> <558AE7B6F747797A2A8B79BD@lewes.staff.uscs.susx.ac.uk> Message-ID: At 11:21 AM +0100 2006-08-10, Ian Eiloart wrote: > It's not necessary to understand all interpretations. There are a few > codes that mean the remote address isn't available. When we see any > other code, we should not count the bounce against the specific > address, because the error isn't related to that address. As I said, show me the code, and then show me all the actual bounces and how they were interpreted by the code. I'd like to see how closely reality actually hews to the RFCs. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Fri Aug 11 13:14:22 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Fri, 11 Aug 2006 12:14:22 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <44DBCA54.6080400@infohazard.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> Message-ID: <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> I'm taking this back on list, since I think the principles are relevant to Mailman. --On 10 August 2006 17:07:48 -0700 Jeff Schnitzer wrote: > Ah, you want to reject /senders/ at SMTP time, not recipients. Ignore my > last message. > > There's no reason why SubEtha can't bounce mail from unknown senders. It > would not be difficult to do*, and may show up in a future release. > However, it's not the universally appropriate solution. Yes, I know. I filed a bug report, and was told it was a feature not a problem. That's when I stopped considering using the s/w. This was a couple of months ago. If the situation changes, then it would be worth considering again. > I see three > different solutions to this problem (mail from unknown recipients): > > 1) Reject it at SMTP time. Legitimate senders are notified that their > mail was not delivered but there is no opportunity to auto-moderate the > message or any opportunity for list administrator to moderate the > message. Great for spam rejection though. > > 2) Hold the message and send the envelope sender instructions to > auto-moderate. Very friendly to legitimate users but bad for the > recipients of joe-jab attacks. However, using SPF makes this a > non-issue. I'm not anti-spf in principle - unlike Brad. However, it's not widely deployed so it can't be relied on in this case. > Using this on a relatively isolated network or behind heavy > spam control makes this a non-issue. Well, there wouldn't be much point putting an MLM server on an isolated network. Not for us, where inter-institutional collaboration is very important. > 3) Hold the message silently, sending no instructions for > auto-moderation. This is not so friendly for legitimately confused users > whose messages go into a vacuum, but the ideal circumstance for > announce-only lists which *always* moderate all messages. Of course, > this is just fine for spam control. Yes, that sucks, but that's where we're left with Mailman. In fact, it's one of the main reasons I'm looking for a MLM that does (1). I can't think of any case in which (3) is better than (1). > There are perfectly legitimate reasons to use any of these 3 approaches. > We started off with #2, and will probably eventually accommodate all > three. I'm sorry if it's not on a schedule that makes you happy, but we > certainly do accept patches. > > * This depends on what sort of MTA you are using. If SubEtha is directly > receiving public input, it's trivial. If inbound mail is being relayed > through another MTA, it gets a lot more complicated. The problem is not > a design issue in SubEtha, it's the antiquated design of all commonly > used MTAs. Anyone who wants to integrate *anything* with > Postfix/Exim/Sendmail/Qmail and friends will be frustrated - they just > don't provide easy hooks into the SMTP exchange. No, it's very easy common to do call forwards with Exim. That would tell me when the list won't accept the sender. So, if Brad's right about Postfix and Sendmail, that just leaves Qmail and friends. Never mind. -- Ian Eiloart IT Services, University of Sussex From barry at python.org Fri Aug 11 22:35:41 2006 From: barry at python.org (Barry Warsaw) Date: Fri, 11 Aug 2006 16:35:41 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> Message-ID: <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ian, I'm not sure what kind of feedback you're looking for, but this one comment stood out to me: On Aug 11, 2006, at 7:14 AM, Ian Eiloart wrote: >> 2) Hold the message and send the envelope sender instructions to >> auto-moderate. Very friendly to legitimate users but bad for the >> recipients of joe-jab attacks. However, using SPF makes this a >> non-issue. > > 3) Hold the message silently, sending no instructions for >> auto-moderation. This is not so friendly for legitimately >> confused users >> whose messages go into a vacuum, but the ideal circumstance for >> announce-only lists which *always* moderate all messages. Of course, >> this is just fine for spam control. > > Yes, that sucks, but that's where we're left with Mailman. In fact, > it's > one of the main reasons I'm looking for a MLM that does (1). I > can't think > of any case in which (3) is better than (1). Although I have not implemented it yet, Mailman 2.2 will definitely get auto-moderation. IOW, should a non-member send a message to a mailing list, and if that mailing list is so configured, Mailman will hold the message and send a message to the From address asking for verification of the post. I'm assuming this is what you mean by "auto-moderation". I'm not worried about joe-jobbing because Mailman could easily send just one auto-moderation message per unit of time, or number of posts to limit any backspamming. There are issues related to how long you want to hold such auto-moderated posts and such, but I think those are all manageable. There's also the question of how long you want a verified non-member to be able to post to a list, but again, this is doable (hopefully without introducing a bajillion new admin knobs). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRNzqInEjvBPtnXfVAQIviwP/TtiHJRvZDOgpGjuK5NS6bmugPZajPLfo 6gYWLWB+e6jdCRrlIOfkYN27lR1NovGzhbZwBP8q3l84aDeNCvyMscCOvaZUjv8m sFbYe2zDYzYG4yjeuY1XFlfXl2PVlgAwh1LQZ3PyJSq08JltTxYG4X+wTzDJOEJZ KWDor5pRDJE= =knT6 -----END PGP SIGNATURE----- From brad at stop.mail-abuse.org Sat Aug 12 02:52:06 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Fri, 11 Aug 2006 19:52:06 -0500 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> Message-ID: At 4:35 PM -0400 2006-08-11, Barry Warsaw wrote: > Although I have not implemented it yet, Mailman 2.2 will definitely > get auto-moderation. IOW, should a non-member send a message to a > mailing list, and if that mailing list is so configured, Mailman will > hold the message and send a message to the From address asking for > verification of the post. I'm assuming this is what you mean by > "auto-moderation". I'm confused. Unless I'm misunderstanding what you're talking about, Mailman 2.1.x already does this today. You try to post to a list that is restricted to subscribers only, and then your message may be rejected, or you may get a message saying that the post is being held for moderation, or it may get silently thrown away, etc.... It all depends on how the listowner has configured things, but to this level, this kind of thing works today. > I'm not worried about joe-jobbing because Mailman could easily send > just one auto-moderation message per unit of time, or number of posts > to limit any backspamming. Okay, now that's different. This actually clicks in very well with the recent article I wrote on fighting spam for publication on the LOPSA website, because backscatter has become one of my biggest hot buttons lately. Putting an intelligent limiter on the potential causes of backscatter (like no more than one notice per recipient per day, or whatever), brings it down into the realm of what I would consider to be less than ideal but at least below the threshold of "totally unacceptable". -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Mon Aug 14 11:44:11 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 14 Aug 2006 10:44:11 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> Message-ID: --On 11 August 2006 16:35:41 -0400 Barry Warsaw wrote: > > Although I have not implemented it yet, Mailman 2.2 will definitely get > auto-moderation. IOW, should a non-member send a message to a mailing > list, and if that mailing list is so configured, Mailman will hold the > message and send a message to the From address asking for verification > of the post. I'm assuming this is what you mean by "auto-moderation". > > I'm not worried about joe-jobbing because Mailman could easily send just > one auto-moderation message per unit of time, or number of posts to > limit any backspamming. Each mailman installation may be able to do this, and that will help a lot. It won't be able to co-ordinate between installations, but this will certainly be better than the current situation where bounces are unrestricted. Maybe SubEtha permits the same, but I've not heard that. I'm not all that bothered about Mailman doing this, because there's a clear way that the MTA can get the information about what Mailman would bounce - by running a python script at STMP time. At least that's true for Exim. This thread arose because someone claimed that SubEtha didn't suck. My comment was that the developers choice of auto-moderation instead of SMTP time rejection was a missed opportunity. It's probably easier to implement, and more frequently the desired behaviour for a closed list. The developers' didn't seem to think SMTP time rejection was desireable, and I think that sucks. > There are issues related to how long you want > to hold such auto-moderated posts and such, but I think those are all > manageable. There's also the question of how long you want a verified > non-member to be able to post to a list, but again, this is doable > (hopefully without introducing a bajillion new admin knobs). > > - -Barry -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Mon Aug 14 11:53:50 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 14 Aug 2006 10:53:50 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <44DD3F2A.3050404@infohazard.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <44DD3F2A.3050404@infohazard.org> Message-ID: --On 11 August 2006 19:38:34 -0700 Jeff Schnitzer wrote: > Brad Knowles wrote: >> Okay, now that's different. This actually clicks in very well with >> the recent article I wrote on fighting spam for publication on the >> LOPSA website, because backscatter has become one of my biggest hot >> buttons lately. Putting an intelligent limiter on the potential >> causes of backscatter (like no more than one notice per recipient per >> day, or whatever), brings it down into the realm of what I would >> consider to be less than ideal but at least below the threshold of >> "totally unacceptable". > > Ok, I just checked this 5-minute fix into SubEtha. Happier, Ian? > > Jeff Yes, that's great! I'd guess that this is 90-99% improvement. In fact, I'd probably use that feature for my local domain (sussex.ac.uk), which is virtually spam free because I only accept from that domain if it's been authenticated, or locally submitted, or has a header that was added by our MSA servers. In fact, the only spam I get into my "local" mailbox is Mailman moderation requests. I'd still want to reject at SMTP time for non-local domains, though. -- Ian Eiloart IT Services, University of Sussex From barry at python.org Mon Aug 14 14:42:00 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 08:42:00 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> Message-ID: <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 11, 2006, at 8:52 PM, Brad Knowles wrote: > At 4:35 PM -0400 2006-08-11, Barry Warsaw wrote: > >> Although I have not implemented it yet, Mailman 2.2 will definitely >> get auto-moderation. IOW, should a non-member send a message to a >> mailing list, and if that mailing list is so configured, Mailman >> will >> hold the message and send a message to the From address asking for >> verification of the post. I'm assuming this is what you mean by >> "auto-moderation". > > I'm confused. Unless I'm misunderstanding what you're talking about, > Mailman 2.1.x already does this today. You try to post to a list > that is restricted to subscribers only, and then your message may be > rejected, or you may get a message saying that the post is being held > for moderation, or it may get silently thrown away, etc.... It all > depends on how the listowner has configured things, but to this > level, this kind of thing works today. Today, held messages still have to be approved by the moderator. What I propose is to allow posters to self-moderate, simply by verifying that their address is real. This probably means a clickable link and (maybe) a header cookie for replying. Think Gmane's auto-moderation approach. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROBvmHEjvBPtnXfVAQJogQQAqMF/7HmCBTMmeEnuwwBAjf+4Ny8n7gpG 8IdqTllrfHcdlwD39NIDC9v+DTczk6xoS3AbyOp5oBFvmk3ZZicoNQXaO/FwuXoE v3SZxrC7axBaPAMtssv0qsBCA8XHt5lCmNU+pvKrSw/4y7sKd3erj+309x0kqBDr 8hPgB8eHjK8= =6Qk7 -----END PGP SIGNATURE----- From barry at python.org Mon Aug 14 14:46:18 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 08:46:18 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 14, 2006, at 5:44 AM, Ian Eiloart wrote: > This thread arose because someone claimed that SubEtha didn't suck. My > comment was that the developers choice of auto-moderation instead > of SMTP > time rejection was a missed opportunity. It's probably easier to > implement, > and more frequently the desired behaviour for a closed list. The > developers' didn't seem to think SMTP time rejection was > desireable, and I > think that sucks. Certainly given sufficient hooks in the MTA, you might be able to make various decisions about the acceptability of a message at SMTP time, although it depends on where in the SMTP dialog you want to hook in. There's no magic in Mailman that would prevent that -- a bit of Python would do the trick. It's not something I'd personally develop because by definition, integration is highly MTA dependent, but I'd accept contributions and would be willing to improve the Mailman infrastructure to make things easier (e.g. see some of the re-org I'm doing in the 2.2 branch to move functionality out of scripts and into the Mailman package so that 3rd party Python code can get to it). - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROBwmnEjvBPtnXfVAQINwAQAgrRAw/HFT3XaX2NlnFLwUm9xyXtSGE5m UBb+uHOPynDZyLjQq4DLM/nHOQ0QNN3a2mOIxYZlJpvcvy11HQEKudhpRoVDxbET WHau682F+KFMH/QBHN7vK9MJlHT7h+G+ZwnBteP/AQuxEPG58TmKtDgLGGMaEKem 97fL4Xos250= =g+pK -----END PGP SIGNATURE----- From iane at sussex.ac.uk Mon Aug 14 15:44:27 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 14 Aug 2006 14:44:27 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> Message-ID: <594B993C53E63C020BE43BF5@lewes.staff.uscs.susx.ac.uk> --On 14 August 2006 08:42:00 -0400 Barry Warsaw wrote: >> I'm confused. Unless I'm misunderstanding what you're talking about, >> Mailman 2.1.x already does this today. You try to post to a list >> that is restricted to subscribers only, and then your message may be >> rejected, or you may get a message saying that the post is being held >> for moderation, or it may get silently thrown away, etc.... It all >> depends on how the listowner has configured things, but to this >> level, this kind of thing works today. > > Today, held messages still have to be approved by the moderator. What I > propose is to allow posters to self-moderate, simply by verifying that > their address is real. This probably means a clickable link and (maybe) > a header cookie for replying. Think Gmane's auto-moderation approach. > So, when someone first posts to a closed list, they're automatically invited to join it if they're not already a member? Provided the list doesn't require moderator approval for joining, I suppose? -- Ian Eiloart IT Services, University of Sussex From iane at sussex.ac.uk Mon Aug 14 15:49:45 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Mon, 14 Aug 2006 14:49:45 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> Message-ID: <45C2996A2637582E4B5BE88E@lewes.staff.uscs.susx.ac.uk> --On 14 August 2006 08:46:18 -0400 Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 14, 2006, at 5:44 AM, Ian Eiloart wrote: > >> This thread arose because someone claimed that SubEtha didn't suck. My >> comment was that the developers choice of auto-moderation instead >> of SMTP >> time rejection was a missed opportunity. It's probably easier to >> implement, >> and more frequently the desired behaviour for a closed list. The >> developers' didn't seem to think SMTP time rejection was >> desireable, and I >> think that sucks. > > Certainly given sufficient hooks in the MTA, you might be able to make > various decisions about the acceptability of a message at SMTP time, > although it depends on where in the SMTP dialog you want to hook in. After "RCPT TO". At that point, if a person isn't allowed to post to the list, the best thing to do is to reject the message. Of the lists I manage, 99% of attempts to post to the list by non-members are spam. Auto-moderation, as you've described, would make sense for some of the lists that I host. > There's no magic in Mailman that would prevent that -- a bit of Python > would do the trick. > > It's not something I'd personally develop because by definition, > integration is highly MTA dependent, but I'd accept contributions and > would be willing to improve the Mailman infrastructure to make things > easier (e.g. see some of the re-org I'm doing in the 2.2 branch to move > functionality out of scripts and into the Mailman package so that 3rd > party Python code can get to it). One thing that would make integration easier, would be a script bin/may_post (or something), which takes a list name (ideally qualified with domain) and sender address, and returns true if the sender address is allowed to post, and false otherwise. > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (Darwin) > > iQCVAwUBROBwmnEjvBPtnXfVAQINwAQAgrRAw/HFT3XaX2NlnFLwUm9xyXtSGE5m > UBb+uHOPynDZyLjQq4DLM/nHOQ0QNN3a2mOIxYZlJpvcvy11HQEKudhpRoVDxbET > WHau682F+KFMH/QBHN7vK9MJlHT7h+G+ZwnBteP/AQuxEPG58TmKtDgLGGMaEKem > 97fL4Xos250= > =g+pK > -----END PGP SIGNATURE----- -- Ian Eiloart IT Services, University of Sussex From bob at nleaudio.com Mon Aug 14 16:27:17 2006 From: bob at nleaudio.com (Bob Puff) Date: Mon, 14 Aug 2006 10:27:17 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> Message-ID: <20060814142549.M53817@nleaudio.com> This should definitely be configurable, if implemented. I don't have any lists that would benefit by this... in fact, this is not desirable on any of my lists. Bob ---------- Original Message ----------- From: Barry Warsaw To: Brad Knowles Cc: Mailman-Developers at python.org, James Ralston , Jon Scott Stevens , Jeff Schnitzer Sent: Mon, 14 Aug 2006 08:42:00 -0400 Subject: Re: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 11, 2006, at 8:52 PM, Brad Knowles wrote: > > > At 4:35 PM -0400 2006-08-11, Barry Warsaw wrote: > > > >> Although I have not implemented it yet, Mailman 2.2 will definitely > >> get auto-moderation. IOW, should a non-member send a message to a > >> mailing list, and if that mailing list is so configured, Mailman > >> will > >> hold the message and send a message to the From address asking for > >> verification of the post. I'm assuming this is what you mean by > >> "auto-moderation". > > > > I'm confused. Unless I'm misunderstanding what you're talking about, > > Mailman 2.1.x already does this today. You try to post to a list > > that is restricted to subscribers only, and then your message may be > > rejected, or you may get a message saying that the post is being held > > for moderation, or it may get silently thrown away, etc.... It all > > depends on how the listowner has configured things, but to this > > level, this kind of thing works today. > > Today, held messages still have to be approved by the moderator. > What I propose is to allow posters to self-moderate, simply by > verifying that their address is real. This probably means a > clickable link and (maybe) a header cookie for replying. Think > Gmane's auto-moderation approach. > > - -Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (Darwin) > > iQCVAwUBROBvmHEjvBPtnXfVAQJogQQAqMF/7HmCBTMmeEnuwwBAjf+4Ny8n7gpG > 8IdqTllrfHcdlwD39NIDC9v+DTczk6xoS3AbyOp5oBFvmk3ZZicoNQXaO/FwuXoE > v3SZxrC7axBaPAMtssv0qsBCA8XHt5lCmNU+pvKrSw/4y7sKd3erj+309x0kqBDr > 8hPgB8eHjK8= > =6Qk7 > -----END PGP SIGNATURE----- > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/bob%40nleaudio.com > > Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp ------- End of Original Message ------- From barry at python.org Mon Aug 14 20:04:57 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 14:04:57 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <594B993C53E63C020BE43BF5@lewes.staff.uscs.susx.ac.uk> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> <594B993C53E63C020BE43BF5@lewes.staff.uscs.susx.ac.uk> Message-ID: <68E21E8C-C09F-44C2-88A1-BD4AC52C4FCE@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 14, 2006, at 9:44 AM, Ian Eiloart wrote: > So, when someone first posts to a closed list, they're automatically > invited to join it if they're not already a member? Provided the list > doesn't require moderator approval for joining, I suppose? My thought was that you'd have an option to allow non-members to post to the list after email verification. If verification is via web page, we can give the poster the option to become a member at the same time. We probably don't want to do that via mail-back verification (just because there's no good way to make this optional in an email message). Say a non-member poster auto-verified, and chose to join the list at the same time, then if moderator approval was required for subscriptions, they'd have to go through that dance in order to join. But they wouldn't have to do a confirmation dance because they'd essentially already confirmed their email address. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROC7SnEjvBPtnXfVAQJstwP+KTnTsvwlB1PvM53Vwt35OCg2Ny+aqaOi 5ngbgyfcrtT5sJUUnU17jd5187uwfHWw/mSjll0yWnMJMYmiW/ClwdxUb3nbyf4t TDIyj1srdXfOZkpiM55+fbxptxAkzOcChEqi8VNf4qe3N71depGklHeqpIWZb9RZ laU/2NTI/3E= =7K1n -----END PGP SIGNATURE----- From barry at python.org Mon Aug 14 20:06:06 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 14:06:06 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <45C2996A2637582E4B5BE88E@lewes.staff.uscs.susx.ac.uk> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <45C2996A2637582E4B5BE88E@lewes.staff.uscs.susx.ac.uk> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 14, 2006, at 9:49 AM, Ian Eiloart wrote: > One thing that would make integration easier, would be a script > bin/may_post (or something), which takes a list name (ideally > qualified > with domain) and sender address, and returns true if the sender > address is > allowed to post, and false otherwise. Why don't you code something up and submit it here? :) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROC7jnEjvBPtnXfVAQI9iwP+JfVMfCZFfd5+cZ6sShF4qXv5YPlwwsTL SmIgCy59P8n39CtJrw/trc1iRB2eLNO/5vKO4ulUXaim1E1CukBr3QaPvMjzQMNH hlmPQliZOwirBQb865RGVr3VQSD28QklAfMNavsbzP+js8skEbz50wGeHGVyayKP wr/2v+is4Lw= =XFWU -----END PGP SIGNATURE----- From barry at python.org Mon Aug 14 20:06:48 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 14:06:48 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <20060814142549.M53817@nleaudio.com> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> <20060814142549.M53817@nleaudio.com> Message-ID: <47E5EFB4-9BA5-472B-B50B-4FB7834DC0E6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 14, 2006, at 10:27 AM, Bob Puff wrote: > This should definitely be configurable, if implemented. I don't > have any > lists that would benefit by this... in fact, this is not desirable > on any of > my lists. Yes, absolutely. It would probably be used on most public discussion lists I'm involved with. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROC7uHEjvBPtnXfVAQJTAgQAhl0LU/3nS6XLCCFhIlQ7NW3r5LLKJpwN bZ+ZdlsNj3sQvu/cGDALic7ACil9HPCOkXaO+mCKeKbLzHdF/UMCMmueX7/z29KS y/MbZDV+2qlB1s3kl6q/7tCKJis64Th555tcCT6yVCxmIMeVeb2z64hUc14tDlZJ 0E12ellRne4= =NvZv -----END PGP SIGNATURE----- From bob at nleaudio.com Mon Aug 14 22:40:41 2006 From: bob at nleaudio.com (Bob Puff@NLE) Date: Mon, 14 Aug 2006 16:40:41 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <66B2DF88-F296-4E96-946D-29AD63BC4097@latchkey.com> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> <20060814142549.M53817@nleaudio.com> <66B2DF88-F296-4E96-946D-29AD63BC4097@latchkey.com> Message-ID: <44E0DFC9.70308@nleaudio.com> Hi Jon, No, I want them to get a message saying they can't do that. The majority of my lists are either one-way announcements or private discussion groups. On the announcement lists, I just drop the messages. They get a lot of spam. Bob Jon Scott Stevens wrote: > On Aug 14, 2006, at 7:27 AM, Bob Puff wrote: > >> This should definitely be configurable, if implemented. I don't have >> any >> lists that would benefit by this... in fact, this is not desirable on >> any of >> my lists. >> >> Bob > > > I'm curious... every time someone sends email to one of your lists from > an address which is not subscribed (ie: their home email or something), > you want to moderate it for them? Wouldn't you rather do less work if > they could self moderate it themselves? > > jon > From bob at nleaudio.com Tue Aug 15 01:02:01 2006 From: bob at nleaudio.com (Bob Puff@NLE) Date: Mon, 14 Aug 2006 19:02:01 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <31C4D2A2-9184-43A2-B5C4-02F30444F871@latchkey.com> References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> <20060814142549.M53817@nleaudio.com> <66B2DF88-F296-4E96-946D-29AD63BC4097@latchkey.com> <44E0DFC9.70308@nleaudio.com> <31C4D2A2-9184-43A2-B5C4-02F30444F871@latchkey.com> Message-ID: <44E100E9.7060204@nleaudio.com> Jon Scott Stevens wrote: > Since the user doesn't have permission to post to the list, the users > message would just get held in the moderation queue and you could > safely ignore it until the queue is auto deleted after a period of > time. The idea being that if the moderation queue only has spam > (because members are given a chance to self moderate if need be... and > the reason for this discussion), then you pretty much just ignore the > queue entirely. This makes the list admin's job trivial. So, even in > your configuration, you would actually want this auto-moderation feature. > > jon Hi Jon, No, I don't want non-members ever being able to post a message. This did happen by accident to one 5000+ member announce-only list, and all hell broke loose! Bob From gail.trac at verizon.net Mon Aug 14 16:27:04 2006 From: gail.trac at verizon.net (gail.trac at verizon.net) Date: Mon, 14 Aug 2006 09:27:04 -0500 (CDT) Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce Message-ID: <5332385.1970251155565625062.JavaMail.root@vms071.mailsrvcs.net> >Today, held messages still have to be approved by the moderator. >What I propose is to allow posters to self-moderate, simply by >verifying that their address is real. This probably means a >clickable link and (maybe) a header cookie for replying. Think >Gmane's auto-moderation approach. >- -Barry I do hope this is meant to be an option the list administrator can set if they want to allow it, not that this would happen for all lists? We're looking to have a totally moderated list, with the moderator's ability to edit certain things out, not change the member's content, but eliminating things like over quoting, and cleaning up format issues too. Will that no longer work? Gail From barry at python.org Tue Aug 15 03:07:24 2006 From: barry at python.org (Barry Warsaw) Date: Mon, 14 Aug 2006 21:07:24 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce In-Reply-To: <5332385.1970251155565625062.JavaMail.root@vms071.mailsrvcs.net> References: <5332385.1970251155565625062.JavaMail.root@vms071.mailsrvcs.net> Message-ID: <59465A46-A78E-402E-93D4-3BAF7C758CD2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 14, 2006, at 10:27 AM, wrote: >> Today, held messages still have to be approved by the moderator. >> What I propose is to allow posters to self-moderate, simply by >> verifying that their address is real. This probably means a >> clickable link and (maybe) a header cookie for replying. Think >> Gmane's auto-moderation approach. > > I do hope this is meant to be an option the list administrator can > set if they > want to allow it, not that this would happen for all lists? Most definitely. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROEeUXEjvBPtnXfVAQLe2QQAmvd9ESLhjVfujPD03nKWyZCMmakhOUUJ zEm4sfBvHWqYXNYh7GhO6b+K0KMy6CnkVAOqsesIzIoSoWEv7icNJPkqURjUrBLM H8kLvZ4Seg+7R1+dAisyVejNNTF2nRVSho4QmH42JKFEX5xcx2xKY9T9CIAX9su+ feN0L+NJeTY= =iNt9 -----END PGP SIGNATURE----- From jwblist3 at olympus.net Tue Aug 15 07:59:57 2006 From: jwblist3 at olympus.net (John W. Baxter) Date: Mon, 14 Aug 2006 22:59:57 -0700 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <0F5347B9-5FC5-400E-8042-D6D52CF5CAE6@python.org> Message-ID: On 8/14/06 5:42 AM, "Barry Warsaw" wrote: > Today, held messages still have to be approved by the moderator. > What I propose is to allow posters to self-moderate, simply by > verifying that their address is real. This probably means a > clickable link and (maybe) a header cookie for replying. Think > Gmane's auto-moderation approach. Unfortunately, the would-be posters then have to be notified of the message status. Thus, while you're reducing moderator workload, the backscatter problem isn't solved. Unfortunately, we know MTAs are hard to write (Exim is still evolving; Postfix took much longer to write than the author expected; sendmail will never be finished). Mailing list managers are hard to write (Mailman is still evolving). So an integrated MTA/MLM would be hard to write (it wouldn't need all the bells and whistles of a full MTA, and would simplify some of the MUA's problems, so the difficulty is probably less than the sum of the difficulties, but still probably more than either alone). (And a newly-written thing doing SMTP would be insecure.) So aside from ruining email, the spammers have ruined email mailing lists. Perhaps irretrievably (at my age of 67, certainly irretrievably in my working lifetime). None of which means it shouldn't be tried, although perhaps it should be tried in the world of whatever comes along to provide a working replacement for SMTP. --John From brad at stop.mail-abuse.org Tue Aug 15 10:40:53 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 15 Aug 2006 03:40:53 -0500 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: Message-ID: At 10:59 PM -0700 2006-08-14, John W. Baxter wrote: > Unfortunately, the would-be posters then have to be notified of the message > status. Thus, while you're reducing moderator workload, the backscatter > problem isn't solved. No, it's not solved. However, by putting a semi-intelligent time limiter on the thing (i.e., no more than one response per address per day, or somesuch), the backscatter problem is at least contained to a more tolerable level. And this does get back to the balance thing that I was taking about earlier. If doing your best to make sure that people know that their message was rejected, or held for moderation, or whatever, is more important to you (and your community), then you've got the option to make those sorts of things happen. If eliminating all possibility of backscatter is more important, you've got the option to do that, too. The point here is to increase your options available to you, and to also try to help reduce the load on list moderators and list owners to a more tolerable level. At least, that's the idea. I'm hoping that the reality will live up to this theory. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From barry at python.org Tue Aug 15 14:19:35 2006 From: barry at python.org (Barry Warsaw) Date: Tue, 15 Aug 2006 08:19:35 -0400 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: Message-ID: <36FB0DAA-4D51-4BE4-98C5-962EE5ADE69A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 15, 2006, at 1:59 AM, John W. Baxter wrote: > On 8/14/06 5:42 AM, "Barry Warsaw" wrote: > >> Today, held messages still have to be approved by the moderator. >> What I propose is to allow posters to self-moderate, simply by >> verifying that their address is real. This probably means a >> clickable link and (maybe) a header cookie for replying. Think >> Gmane's auto-moderation approach. > > Unfortunately, the would-be posters then have to be notified of the > message > status. Thus, while you're reducing moderator workload, the > backscatter > problem isn't solved. But I think it can be mitigated. You simply don't send a verification for every posting your holding. Maybe you send a summary every three days until the messages expire unverified. > So an integrated MTA/MLM would be hard to write (it wouldn't need > all the > bells and whistles of a full MTA, and would simplify some of the MUA's > problems, so the difficulty is probably less than the sum of the > difficulties, but still probably more than either alone). (And a > newly-written thing doing SMTP would be insecure.) Mailman won't be that integrated MTA/MLM, although it may have tools that help integrate Mailman with the most popular MTAs. I have a clear picture of what I see Mailman doing and it's not the MTAs job or SpamAssassin's job. It's only barely doing Hypermail's job (and that's debatable). > So aside from ruining email, the spammers have ruined email mailing > lists. > Perhaps irretrievably (at my age of 67, certainly irretrievably in my > working lifetime). > > None of which means it shouldn't be tried, although perhaps it > should be > tried in the world of whatever comes along to provide a working > replacement > for SMTP. I tend to be more sanguine about things. I'm younger than you but I've been around for long enough to have heard about the death of the internet/arpanet for 25 years. It hasn't happened yet and I don't think email and SMTP is going away any time soon. Maybe it should. Maybe all the kids will gravitate toward other modes of communication and leave us dinosaurs to our spam riddled 20th century telegraphs. Or maybe we'll stay just barely ahead of the spammers enough to eek out the benefits of email and mailing lists for another 20 years. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROG73HEjvBPtnXfVAQIKowQApwRs3Q5+pQaDG8W1fMWi2hcum8oYKmhQ jBLONMPnpWJMIdvNLiuMfNmvSAU1MUCU2xVkia1lxSGZOVf/2+grrObh0GfnR/Kk O4+Gnj1W4hIKCzE/hKLYDYzKxFc1liOXZ2XL3rb9Y67V0mJPw9UQokKW40+R01Gn zcGgFDe8cg4= =qasI -----END PGP SIGNATURE----- From brad at stop.mail-abuse.org Tue Aug 15 20:22:49 2006 From: brad at stop.mail-abuse.org (Brad Knowles) Date: Tue, 15 Aug 2006 13:22:49 -0500 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: <36FB0DAA-4D51-4BE4-98C5-962EE5ADE69A@python.org> References: <36FB0DAA-4D51-4BE4-98C5-962EE5ADE69A@python.org> Message-ID: At 8:19 AM -0400 2006-08-15, Barry Warsaw wrote: > I tend to be more sanguine about things. I'm younger than you but > I've been around for long enough to have heard about the death of the > internet/arpanet for 25 years. It hasn't happened yet and I don't > think email and SMTP is going away any time soon. We're certainly getting there for some people. I found out the other night that my Mom no longer bothers doing e-mail. Okay, she's 62, retired six months early due to medical problems (terminal cancer), but she's still got a few good months left and she doesn't want to waste them trying to fight spam in her mailbox. So, she just reads most of the time. My own spam load is around 90-99%, depending on how bad the day is. My ISP routes all their mail for their customers through Postini, and they catch 90% of that, but that still leaves a lot for the ISP to deal with. So, they set up their own secondary anti-spam handling system, which is still as large or larger than the entire rest of the mail system put together. And I still get an annoying amount of spam that gets through to my client, which also has anti-spam features integrated. I can certainly see why many people would get to the point where they start feeling like e-mail no longer has any real value. I certainly feel that way about most USENET newsgroups I know of, and for the same reasons. > Maybe all the kids will gravitate toward other modes of communication > and leave us dinosaurs to our spam riddled 20th century telegraphs. They already have. It's called IM, chat, or txtng -- depending on the exact platform. Many times I've said that e-mail is the only universal mission-critical platform, but I've also said that each organization may have their own mission-critical applications on top of that. AOL is no different. When I was the Sr. Internet Mail Administrator for AOL, we had only two mission-critical applications -- e-mail and chat. If they weren't available, then most customers would just leave, because there wasn't much of anything else that they wanted to do. And "spim" is already a major problem, or so I hear. I haven't heard of "spat" or "sptxt" being much of an issue, but I'm sure that they'll figure out a way to abuse those systems as well. Thanks to Dateline NBC and Stone Phillips, we have certainly seen way more than we ever wanted to know about how predators use IM to lure kids into abusive situations, and I guess that would probably be the worst form of "spim". > Or maybe we'll stay just barely ahead of the spammers enough to eek > out the benefits of email and mailing lists for another 20 years. I think we'll try, and for some people we will succeed, but my fear is that more and more people are going to start giving up on e-mail and will switch to alternative communication methods. Those methods are likely to be less convenient because if it's too convenient for us then it will probably be much too convenient for spammers. -- Brad Knowles, "Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety." -- Benjamin Franklin (1706-1790), reply of the Pennsylvania Assembly to the Governor, November 11, 1755 Founding Individual Sponsor of LOPSA. See . From iane at sussex.ac.uk Tue Aug 15 20:37:01 2006 From: iane at sussex.ac.uk (Ian Eiloart) Date: Tue, 15 Aug 2006 19:37:01 +0100 Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce processing In-Reply-To: References: <44DBBE7D.5080207@infohazard.org> <44DBCA54.6080400@infohazard.org> <8CD46999E4E1C4CAB0D05619@lewes.staff.uscs.susx.ac.uk> <9D628A0A-DAB2-4DB0-8611-4B47C054047B@python.org> <45C2996A2637582E4B5BE88E@lewes.staff.uscs.susx.ac.uk> Message-ID: <0AE01181B2F7E2148EAB5B92@lewes.staff.uscs.susx.ac.uk> --On 14 August 2006 14:06:06 -0400 Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Aug 14, 2006, at 9:49 AM, Ian Eiloart wrote: > >> One thing that would make integration easier, would be a script >> bin/may_post (or something), which takes a list name (ideally >> qualified >> with domain) and sender address, and returns true if the sender >> address is >> allowed to post, and false otherwise. > > Why don't you code something up and submit it here? :) > > - -Barry I started to write that I've no python coding experience. Well, about 3 lines because php can't do "utf-something or other". Then I thought, well it's about time I got some. I had hacked up a shell script using the existing Mailman scripts, but that was far too inefficient. Instead I've hacked up the attached. It started life as list_config, but hopefully I've not left much trace of that. The second issue below ***MUST*** be resolved before using this script with an MTA. The attached script takes these arguments: -o --outputfile FILE_PATH can be used to specify logging of denies. use '-' to log to stdout -v --verbose causes logging of all results, allows as well as denies. -h --help prints help -s --sender EMAIL_ADDRESS is required The script applies these tests, printing 'allow' or 'deny' to std out on the first match. allow list owners allow list moderators allow members of accept_these_nonmembers deny members of reject_these_nonmembers if generic_nonmember_action is 'reject': allow members to post deny non-members allow by default These issues are outstanding: ---------------------------- On allow, I say "return 1" on deny I say "return 0". I'm not sure whether that's correct. Actually, I think I want the script to succeed every time, so it can't be. I've not figured out how to do a pattern match so accept_these_nonmembers and reject_these_nonmembers are only tested for exact string matches. This *****needs to be fixed***** for accept_these_nonmembers, otherwise some won't be permitted to post. It'd be nice to log to syslog, but the MTA could take care of that. It might be nice to say 'hold' or 'discard' where appropriate. It's often sensible to reject rather than discard a message, for example. The list's nonmember_rejection_notice isn't used here. It could be returned instead of 'deny' for the MTA to construct a rejection string with. I've hard-coded '2' as the 'reject' key to generic_nonmember_action, which is sinful. -- Ian Eiloart IT Services, University of Sussex -------------- next part -------------- #! /local/bin/python # # Copyright (C) 1998-2003 by the Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. """Find out whether list will reject a message from sender. Usage: check_sender.py [options] -s sender listname Options: --outputfile filename -o filename optionally log denys to list file. --sender sender -s sender check whether the sender is allowed to post to the list. --verbose -v log allows as well as denys. --help -h Print this help message and exit. The option -s is required. """ import sys import re import time import getopt from types import TupleType import paths from Mailman import mm_cfg from Mailman import MailList from Mailman import Utils from Mailman import Errors from Mailman.i18n import _ NL = '\n' def usage(code, msg=''): if code: fd = sys.stderr else: fd = sys.stdout print >> fd, _(__doc__) if msg: print >> fd, msg sys.exit(code) def do_check(listname, sender, outfile, verbose): closep = 0 try: if outfile == '-': outfp = sys.stdout else: outfp = open(outfile, 'a') closep = 1 # Open the specified list unlocked, since we're only reading it. try: mlist = MailList.MailList(listname, lock=0) except Errors.MMListError: usage(1, _('No such list: %(listname)s')) # get all the list config info. all this stuff is accessible via the # web interface when = time.strftime('%Y-%m-%d %H:%M:%S',time.localtime()) # always allow the owner to post if sender in mlist.owner : if verbose: print >> outfp, _('''%(when)s %(listname)s owner %(sender)s allowed''') print >> sys.stdout, 'allow' return 1 # always allow moderators to post if sender in mlist.moderator : if verbose: print >> outfp, _('''%(when)s %(listname)s moderator %(sender)s allowed''') print >> sys.stdout, 'allow' return 1 # always allow accept_these_nonmembers to post if sender in mlist.accept_these_nonmembers : if verbose: print >> outfp, _('''%(when)s %(listname)s accept_these_nonmembers %(sender)s allowed''') print >> sys.stdout, 'allow' return 1 # deny reject_these_nonmembers if sender in mlist.reject_these_nonmembers : print >> outfp, _('''%(when)s %(listname)s reject_these_nonmembers %(sender)s allowed''') print >> sys.stdout, 'deny' return 0 # 2 is the key for 'reject', but there's probably a global we should use if mlist.generic_nonmember_action == 2: # regular members rmembers = mlist.getRegularMemberKeys() # digest members dmembers = mlist.getDigestMemberKeys() allmembers = rmembers + dmembers # should lowercase sender if sender.lower() in allmembers: if verbose: print >> outfp, _('''%(when)s %(listname)s member %(sender)s allowed''') print >> sys.stdout, 'allow' return 1 else: print >> outfp, _('''%(when)s %(listname)s member %(sender)s denied''') print >> sys.stdout, 'deny' return 0 print >> outfp, _('''%(when)s %(listname)s unknow %(sender)s allowed by default''') print >> sys.stdout, 'allow' return 1 finally: if closep: outfp.close() def main(): try: opts, args = getopt.getopt( sys.argv[1:], 's:o:vh', ['list=', 'sender=', 'verbose', 'help']) except getopt.error, msg: usage(1, msg) # defaults infile = None outfile = None list = None sender = None verbose = 0 for opt, arg in opts: if opt in ('-h', '--help'): usage(0) elif opt in ('-l', '--list'): list = arg elif opt in ('-s', '--sender'): sender = arg elif opt in ('-o', '--outputfile'): outfile = arg elif opt in ('-v', '--verbose'): verbose = 1 # sanity check if sender is None: usage(1, _('--sender is required')) # get the list name if len(args) <> 1: usage(1, _('List name is required')) listname = args[0].lower().strip() do_check(listname, sender, outfile, verbose) if __name__ == '__main__': main() From gail.trac at verizon.net Tue Aug 15 20:46:05 2006 From: gail.trac at verizon.net (gail.trac at verizon.net) Date: Tue, 15 Aug 2006 13:46:05 -0500 (CDT) Subject: [Mailman-Developers] Fwd: suggested improvement for Mailman's bounce pro Message-ID: <29262854.162781155667565904.JavaMail.root@vms064.mailsrvcs.net> >I can certainly see why many people would get to the point where they >start feeling like e-mail no longer has any real value. I certainly >feel that way about most USENET newsgroups I know of, and for the >same reasons. >Brad Knowles, Eamil has indeed become very frustrating lately, my email client is currently fried and hopefully the guy coming over tonight can figure out how to retrieve it, i've tried all i know and can't get it to open at all. I also get all the spam you've talked about, and the current software we're using to host a list on sends me every piece of spam received for any imaginary address the spam folks make for the domain on top of my more than healthy dose of personal spam of course. Its very annoying and at times its been a good thing I can't quite reach the computer to throw it out into oncoming traffic, however, I work at home so its a vital part of what allows me to do the various jobs from home. Now, I've noticed a marked decrease in the viruses circulating in recent months, and a huge increase in spam at about the same time. A very senior tech guy I respect a great deal has told me that the virus writing jerks have now been hired to write software that spreads spam like they used to spread viruses and I believe it. I told him i was getting the same sort of sinking feeling about the internet hanging on by a thread like i used to get when the major finanacial services applications i worked with were about to crash and he brought that up in response. I used to say, about the virus jerks, that if only they'd put their talents to some good use they could change or rule the world and make a fortune doing it, unfortunately it appears they've now figured out how to make that fortune doing spam instead of something 'good' for the world. Sad isn't it? Gail From barry at python.org Wed Aug 16 14:42:58 2006 From: barry at python.org (Barry Warsaw) Date: Wed, 16 Aug 2006 08:42:58 -0400 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7985] trunk/mailman/messages In-Reply-To: References: Message-ID: <967DB52A-CA19-4F8C-BE6D-FFB40DE467EF@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 16, 2006, at 4:04 AM, tkikuchi at users.sourceforge.net wrote: > Arabic message for mailman-2.2 ... it wasn't there because I used > cp -r from > mailman-2.1 directory and .svn was also copied. :-( Yep, svn merge is probably the best thing to use to do this kind of cross-branch updates. (BTW, very cool to see Apple's Mail display all those Arabic characters!) - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROMS2XEjvBPtnXfVAQIHrQQAh6nYJGjMiFsSI1P2bCmVtDweblPvgs7V RvliCTqbzrfpDKIsxEM4/R5+BXIfaFiQDUxRbCITd1h4ujRAbmWv2QJclL4xIyo8 bhqT04UWhvVodJu5opmrmFVVMxzD+O72BJEu9MM/1fuW8UKFJpBl7m0SK1URR2JD JpGTSwUl0ss= =CS59 -----END PGP SIGNATURE----- From i at mindlace.net Mon Aug 21 04:12:53 2006 From: i at mindlace.net (emf) Date: Sun, 20 Aug 2006 22:12:53 -0400 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7989] branches/soc2006-webui In-Reply-To: References: Message-ID: <44E916A5.9060009@mindlace.net> Sorry for all the property changes. SF.net doesn't seem to let me set repository tags. Any chance someone (barry?) can add .pyc to svn:ignore on the server side? Thanks, ~ethan From i at mindlace.net Mon Aug 21 04:17:29 2006 From: i at mindlace.net (emf) Date: Sun, 20 Aug 2006 22:17:29 -0400 Subject: [Mailman-Developers] development setup? Message-ID: <44E917B9.5060906@mindlace.net> Gentlebeings, As we enter the end of my SoC internship, I still have a long road to hoe to get to a complete UI for Mailman. That being said, I'm definitely going to continue working with this at least until it makes it into a mainline release, and hopefully longer. Here's my quandary: I need to have several mailman installs 'up and running' - on a "I can talk to the pickles" level, anyway. How do I get such a beast out of svn? INSTALL points me to an 'admin' directory that isn't in svn. I had thought of just adding svn_dir to my pythonpath, but I need a configuration file, and in general I would like my code to be verbatim what is needed to run it against a production server. Thanks, ~ethan From i at mindlace.net Mon Aug 21 04:25:24 2006 From: i at mindlace.net (emf) Date: Sun, 20 Aug 2006 22:25:24 -0400 Subject: [Mailman-Developers] Timing, events, runners, oh my Message-ID: <44E91994.1090703@mindlace.net> Gentlebeings, The UI would like to use email for a variety of functions, and it would like some event-based code. I was wondering how much of mailman's existing infrastructure I should use. a.) I would like to allow "email login", where a passwordless user receives an email confirmation link. Whom should I send this as? Should I inject it into mailman's queue? b.) If the UI could be notified when changes occur on the command line or via email, I could do some more intelligent caching. What's the best place to make that happen? c.) I'd like to have events and observers and whatnot; this is down the road a bit, for 'improved' digest support among other things. Would it be inappropriate to have a UIEventsRunner ? ~ethan fremen From barry at python.org Mon Aug 21 04:33:19 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Aug 2006 22:33:19 -0400 Subject: [Mailman-Developers] [Mailman-checkins] SF.net SVN: mailman: [7989] branches/soc2006-webui In-Reply-To: <44E916A5.9060009@mindlace.net> References: <44E916A5.9060009@mindlace.net> Message-ID: <1EA5AB98-4E55-4A80-B6F5-E9B95189EFA2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 20, 2006, at 10:12 PM, emf wrote: > Sorry for all the property changes. SF.net doesn't seem to let me set > repository tags. Any chance someone (barry?) can add .pyc to > svn:ignore > on the server side? I highly suggest adding these to your .subversion/config file instead, so you get the benefit in all your Python projects, not just Mailman. I've asked Greg Stein several times to add it (and .elc :) by default. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROkbb3EjvBPtnXfVAQJmiQP/Z7OCF4QgbFsB3Jv3BAFKcQw1m+XdOJRR OPhPPn7VUtfIhs5e3ch45Sw3rj1/1fQZywwoN9xICxOPlf+b9OY+vzWKiCR7bTIZ svH14Gd5qrvx4MUqcr7R7WNY8X3fuhzIx2KImsEf1ZcM95hqHfYPRnT1QuhOlvXN fBNWAFYcZyo= =7xu6 -----END PGP SIGNATURE----- From barry at python.org Mon Aug 21 05:02:06 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Aug 2006 23:02:06 -0400 Subject: [Mailman-Developers] development setup? In-Reply-To: <44E917B9.5060906@mindlace.net> References: <44E917B9.5060906@mindlace.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Aug 20, 2006, at 10:17 PM, emf wrote: > How do I get such a beast out of svn? INSTALL points me to an 'admin' > directory that isn't in svn. I had thought of just adding svn_dir > to my > pythonpath, but I need a configuration file, and in general I would > like > my code to be verbatim what is needed to run it against a > production server. On the trunk, the admin directory is one up from the source directory. Eventually, the trunk version and branch version of the admin dir will be merged. So you can either check out the dir above the source dir or grab it from SF's viewsvn interface. It's best to do a ./configure; make; make install, then put $prefix/ on your pythonpath. I'd like for Mailman to be runnable out of the source dir eventually, but that doesn't really work well right now. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROkiL3EjvBPtnXfVAQL0FAQAl+upw044DCIVGhQLCGMaiWZrhVRmvimL koSoEZ+o23NWbDBaP685eVswU9zJTXvDRXgzNH2F/CaMYj2rHZ5XFV3V6adLrncl WTFat7q5bBSBDeEGITODyiYL+g9nBZrVd4cFhjo2tSCgWwNxZS3b8xGVxBH7/vMN TR16pM7l+YU= =v2M1 -----END PGP SIGNATURE----- From barry at python.org Mon Aug 21 05:25:37 2006 From: barry at python.org (Barry Warsaw) Date: Sun, 20 Aug 2006 23:25:37 -0400 Subject: [Mailman-Developers] Timing, events, runners, oh my In-Reply-To: <44E91994.1090703@mindlace.net> References: <44E91994.1090703@mindlace.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Since the SoC code is written against the trunk, I'll answer these questions for MM2.2. On Aug 20, 2006, at 10:25 PM, emf wrote: > a.) I would like to allow "email login", where a passwordless user > receives an email confirmation link. Whom should I send this as? > Should > I inject it into mailman's queue? If you want to allow for a simple reply to confirm the action, you'll need to use the same code that uses Pending.py to generate a cookie and a database entry, then send the email from a request address with a cookied from address. See something like MailList.InviteNewMember(). > b.) If the UI could be notified when changes occur on the command line > or via email, I could do some more intelligent caching. What's the > best > place to make that happen? It depends on what you want to be notified of. In general, a Handler module is a good place to have actions occur in response to emails. There isn't a central place to drop things to react to command line actions. > c.) I'd like to have events and observers and whatnot; this is down > the > road a bit, for 'improved' digest support among other things. Would it > be inappropriate to have a UIEventsRunner ? Think of the runners as a way to get some concurrency out of the system without biting the big bitter apple of threads. Runners are focussed on actions involving emails though. Without more detail, I can't say that a UIEventsRunner makes sense, but it's not out of the question. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBROknsnEjvBPtnXfVAQKdbAP+LxeB7qyS1tBp/l083nsaDq2Glny6Qa0g uiFNreJAHJtt0oXrt93lszxJwMHy8u0xBBQXxU9l415g1CtpSWxx7tUk7KsGqUZ5 2swXWX/fcwcqEd/jp/lFDax7aeBdmB9zpxSVuVSI/QAZRaTnhWQ6JhYzuK0tQaXC W336e2v8+bI= =i5MW -----END PGP SIGNATURE----- From i at mindlace.net Mon Aug 21 05:33:04 2006 From: i at mindlace.net (emf) Date: Sun, 20 Aug 2006 23:33:04 -0400 Subject: [Mailman-Developers] SoC Summary Message-ID: <44E92970.8090709@mindlace.net> Gentlebeings, A thousand pardons for not delivering a functional UI by the deadline of my SoC project. I have written a summary of my experiences at: http://wiki.list.org/display/DEV/2006/08/20/Summer+of+Code+summary and I assure you that I will continue to work on this project for the foreseeable future, or at least until it makes it into a main-line release. Thank you all for all the help you've been so far; I expect to continue to rely on it as I move closer to something we can actually use. ~ethan fremen From i at mindlace.net Mon Aug 21 06:27:42 2006 From: i at mindlace.net (emf) Date: Mon, 21 Aug 2006 00:27:42 -0400 Subject: [Mailman-Developers] Security standards? Message-ID: <44E9363E.1090507@mindlace.net> So as I understand it, mm 2.2 is dropping email reminders of passwords. This has made me think that we'd like more support for 'passwordless' manipulation of the UI. I've come up with a few approaches for this, and I'd like to get feedback as to what would be acceptable. Please keep in mind I'll allow administrators to require more authentication than I outline below. * Use case A: an email is a member of a mailing list but has never logged into the interface. I was thinking it would be ok in this context to allow a user-agent to approach the interface and provide only the email address to be "provisionally authenticated"; they would be allowed to manipulate the member's settings. Once they were done doing so, an email would be sent to the address that required clicking on a confirmation link to make the changes active. * Use case B: a user-agent presents an email that has used the interface previously. If the user-agent presents a visitation cookie that was active during the previous manipulation, the user is provisionally authenticated again, and gets a similar confirmation email. If they did *not* have a matching visitation cookie, or present another this-is-really-me token, they would not be allowed to manipulate the interface until they click a email-verification link. * Use case C: Some other code (an upstream process, OpenId server, etc) provides a username for a user. In this case we accept the user as authenticated, and either use our map of username-> email addresses for purposes of determining membership or accept an email_addresses list from the WSGI environment or from the remote server/other process. If only the username is provided, the user is given the opportunity to indicate which email addressess should be associated with that username; once they have done so, verification emails are sent to said addresses, and post reply/link activation mailman considers those emails to be associated with that username. Does this stuff sound reasonable? ~ethan fremen From tandrews at grok.co.za Sun Aug 27 14:05:42 2006 From: tandrews at grok.co.za (Thomas Andrews) Date: Sun, 27 Aug 2006 14:05:42 +0200 Subject: [Mailman-Developers] subscription barrier Message-ID: <20060827120542.GE20898@grok.co.za> Hi, On the asterisk-dev list, we constantly get questions that should have been sent to asterisk-users. (I'm sure the same has happened here.) I have a suggestion which people might like: If one can be required to answer a question correctly during the subscription proces, unnecessary memberships could be blocked. For example, let's say the admin made the question" "What is the value of x in typedef enum { u,v,x,y,z=9 } e; ?" The average non-C programmer would probably get the message and not bother to subscribe. It would also take a bit of trouble to find out the correct answer if they did try. Of course it would be up to the list admin to enable this feature, and choose the question (and it's answer.) A small useful side-effect is that it might help raise the bar against spammers doing scripted list subscriptions. Regards, Thomas From i at mindlace.net Wed Aug 30 02:16:53 2006 From: i at mindlace.net (emf) Date: Tue, 29 Aug 2006 20:16:53 -0400 Subject: [Mailman-Developers] trunk checkins ok? Message-ID: <44F4D8F5.3030208@mindlace.net> Gentlebeings, There are things I'd like to change that seem like they'd be best changed on the trunk; the two examples at the moment are list_names() doesn't talk to the config object properly and I found a python implementation of line-wrapping by a number-theory guy that claims to be at least as filled with fu as the emacs version, which I'd like to substitute for wrap(). Is it ok if I make changes on the trunk? ~ethan