From barry at python.org Mon Mar 14 20:44:04 2005 From: barry at python.org (Barry Warsaw) Date: Mon Mar 14 20:44:09 2005 Subject: [Mailman3-dev] Can you donate a public subversion repository for Pycon sprints? Message-ID: <1110829380.15320.82.camel@geddy.wooz.org> I'm wondering if anybody out there can donate a Subversion repository for use at the upcoming Pycon sprint next week (actually, starting Saturday). I had a machine that I was going to use for this, but it lost its disk over the weekend, and it doesn't look like I'm going to be able to get it back online in time. :( Ideally, someone who will be coming to the sprint would be able to donate a repository, since we'll need to set up access control once we're there and we know who's going to be participating. It will be tough to coordinate this if you're not at the sprint, but if you're willing to be accessible via IM or IRC, we might be able to work around that. I'd like for us to have a separate repository, not tainted by other projects or code. Access can be by http or ssh+svn, whichever you prefer (although the latter requires us to coordinate pubkeys, so that's less desirable). I would send you a dump file of my private repository, and then you'd send me a dump file when the sprints are all done. This will not be the permanent subversion repository for Mailman3. For the long term, we'll either host at SourceForge when they make Subversion available, or I'll host it at my site after I get my machine back online. We just need something we can use for the week of Pycon. If you think you can help out, please respond to me directly. Thanks! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050314/09b1e176/attachment.pgp From barry at python.org Mon Mar 14 23:23:13 2005 From: barry at python.org (Barry Warsaw) Date: Mon Mar 14 23:23:16 2005 Subject: [Mailman3-dev] Re: Can you donate a public subversion repository for Pycon sprints? In-Reply-To: <1110829380.15320.82.camel@geddy.wooz.org> References: <1110829380.15320.82.camel@geddy.wooz.org> Message-ID: <1110838993.16647.12.camel@geddy.wooz.org> On Mon, 2005-03-14 at 14:44, Barry Warsaw wrote: > I'm wondering if anybody out there can donate a Subversion repository > for use at the upcoming Pycon sprint next week (actually, starting > Saturday). We have a winner! John Viega reminds me that we can use list.org! Whodah thunk? :) The machine recently got rebuilt with plenty of disk space and already has svn on it, so we just need to set up the access infrastructure. I should be able to have the repo up before Saturday, and I will send (read-only) instructions when it's available. We'll work out write perms at the sprint. Thanks everyone who offered to help. I hope to see you all at Pycon, if not the sprints. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050314/7b2cfd3b/attachment.pgp From barry at python.org Thu Mar 17 17:32:03 2005 From: barry at python.org (Barry Warsaw) Date: Thu Mar 17 17:32:10 2005 Subject: [Mailman3-dev] Mailman 3 Sprint at Pycon2005 Message-ID: <1111077123.9861.19.camel@geddy.wooz.org> Just a reminder that we're going to do a Mailman 3 sprint again this year at Pycon2005, which starts Saturday. I plan on being there all four sprint days (Sat-Tues) and I hope you will be able to join us. For the meager information on the MM3 sprint see: http://www.python.org/moin/Mailman3Sprint Please sign up on this page if you plan on coming. Remember that the sprint days at Pycon are free; for more information on the conference in general see: http://www.pycon.org I plan on logging into the #mailman channel on irc.freenode.net so if you can't make it and have questions, we'll try to discuss things there, although we might be too busy to pay much attention to irc. I should have more information about the Subversion repository for MM3 later today. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050317/cb4e6ef4/attachment.pgp From barry at python.org Sat Mar 19 16:57:38 2005 From: barry at python.org (Barry Warsaw) Date: Sat Mar 19 16:57:48 2005 Subject: [Mailman3-dev] Mailman 3 subversion repository online Message-ID: <1111247857.11098.6.camel@geddy.wooz.org> The Mailman 3 subversion repository is now on-line. You can check out a read-only copy of the tree from this url: http://svn.list.org You can also visit that url in your browser to view the tree. Write access will be via https://svn.list.org but you must be an authorized user to check out (or check in, of course). Diffs will be going to mailman-checkins@python.org. If you need more information about using Subversion, including downloading clients, see http://subversion.tigris.org. BTW, I will be hanging out on #mailman on irc.freenode.net. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050319/ede4f9ae/attachment.pgp From rmunn at pobox.com Tue Mar 29 07:50:52 2005 From: rmunn at pobox.com (Robin Munn) Date: Tue Mar 29 15:06:34 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? Message-ID: <4248ECBC.8080509@pobox.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 During the PyCon sprints, we created a MailingList object in src/mailman/mlist.py which has __getattr__() and __setattr__() methods to set mailing list attributes such as "max_hold" or "enable_archives". I've been rethinking that decision; I think another method might be slightly better. Instead of using straight attribute access, I think it might be better to use a proxy object. Call it "attr", or maybe just "a". Possibly both could be allowable. Thus to create a non-archived mailing list, one would do something like this under the current scheme: ~ mlist = MailingList(...) ~ mlist.enable_archives = False Whereas what I'm wondering is whether we should do it like this: ~ mlist = MailingList(...) ~ mlist.attr.enable_archives = False Or possibly use a different name than "attr": ~ mlist = MailingList(...) ~ mlist.opts.enable_archives = False The proxy object would be simple: ~ class AttributeFetcher(object): ~ def __init__(self, parent): ~ self.parent = parent ~ def __getattr__(self, attrname): ~ # Same code as is currently in MailingList ~ def __setattr__(self, attrname, attrvalue): ~ # ditto ~ class MailingList(object): ~ . ~ . ~ . ~ def __init__(self): ~ self.attr = AttributeFetcher(self) ~ . ~ . ~ . Pros of what we have now: ~ * Simple. Easy to understand. Easy to read. Cons of what we have now: ~ * Methods and mailing list attributes are living in the same namespace. Could hit collisions. Pros of the second method (mlist.attr.some_option): ~ * Still simple, still easy to understand. ~ * No namespace collisions. Cons of the second method: ~ * More typing involved. ~ * Two dereferences instead of one -- Law of Demeter says this smells. I'm not sure I entirely believe the last con, though: the Law of Demeter is only intended to apply to coupling between distinct objects. Here, AttributeFetcher is a helper class specifically designed for MailingList; it *should* be tightly coupled. What do you think? - -- Robin Munn rmunn@pobox.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCSOy36OLMk9ZJcBQRAgNpAJ9bL94QqLrra1oOyqLj5Pm/gPZk1gCeJixc cBQCDQ3vIkc/lCtdQMFxTcE= =b/X5 -----END PGP SIGNATURE----- From mark at gaiahost.coop Tue Mar 29 16:58:47 2005 From: mark at gaiahost.coop (Mark Bucciarelli) Date: Tue Mar 29 17:35:22 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <4248ECBC.8080509@pobox.com> References: <4248ECBC.8080509@pobox.com> Message-ID: <42496D27.20104@gaiahost.coop> Robin Munn wrote: > ~ * Methods and mailing list attributes are living in the same > namespace. Could hit collisions. I see your point. The list attributes are pulled from a SQL table, so it's possible that someone could add a custom attribute (to the table) and then a later release of mailman could add a new class attribute with that name. That could lead to some amazing bug. But this is the only scenario I could think of; i.e. where a user adds a custom attribute. Seems very low probability and perhaps could be handled by documentation (e.g., all custom attributes should use an "x_" prefix in their name) and an exception or assertion failure in the code that sets the attributes after reading them from the db. I guess that syntax mlist.attr.prop just bugs me. :) By the way, how did you guys do on Tuesday? Regards, Mark From claw at kanga.nu Wed Mar 30 01:28:03 2005 From: claw at kanga.nu (J C Lawrence) Date: Wed Mar 30 01:28:08 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: Message from Mark Bucciarelli of "Tue, 29 Mar 2005 09:58:47 EST." <42496D27.20104@gaiahost.coop> References: <4248ECBC.8080509@pobox.com> <42496D27.20104@gaiahost.coop> Message-ID: <16736.1112138883@kanga.nu> On Tue, 29 Mar 2005 09:58:47 -0500 Mark Bucciarelli wrote: > I see your point. The list attributes are pulled from a SQL table, so > it's possible that someone could add a custom attribute (to the table) > and then a later release of mailman could add a new class attribute > with that name. That could lead to some amazing bug. Having spent the last week looking at just such bugs, yeah. > But this is the only scenario I could think of; i.e. where a user adds > a custom attribute. Users are going to want to do that. They are going to want to implement custom schema with attributes which are then reflected at the SMTP layer, in the web UI, etc etc. They are going to want to do this arbitrarily, and they are going to expect that their code will work across multiple Mailman version releases. They are going to want to be able to be Mailman experts. We should support that. > Seems very low probability and perhaps could be handled by > documentation (e.g., all custom attributes should use an "x_" prefix > in their name) and an exception or assertion failure in the code that > sets the attributes after reading them from the db. That seems like a chunk of complexity for little gain. > I guess that syntax mlist.attr.prop just bugs me. :) Then use a new-class property method? -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw@kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live. From hazmat at objectrealms.net Wed Mar 30 01:42:18 2005 From: hazmat at objectrealms.net (Kapil Thangavelu) Date: Wed Mar 30 02:12:51 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <16736.1112138883@kanga.nu> References: <4248ECBC.8080509@pobox.com> <42496D27.20104@gaiahost.coop> <16736.1112138883@kanga.nu> Message-ID: <2FEA2B52-A0AC-11D9-B064-000393DB3C38@objectrealms.net> On Mar 29, 2005, at 3:28 PM, J C Lawrence wrote: > On Tue, 29 Mar 2005 09:58:47 -0500 > Mark Bucciarelli wrote: > >> I see your point. The list attributes are pulled from a SQL table, so >> it's possible that someone could add a custom attribute (to the table) >> and then a later release of mailman could add a new class attribute >> with that name. That could lead to some amazing bug. > > Having spent the last week looking at just such bugs, yeah. > >> But this is the only scenario I could think of; i.e. where a user adds >> a custom attribute. > > Users are going to want to do that. They are going to want to > implement > custom schema with attributes which are then reflected at the SMTP > layer, in the web UI, etc etc. They are going to want to do this > arbitrarily, and they are going to expect that their code will work > across multiple Mailman version releases. They are going to want to be > able to be Mailman experts. > > We should support that. > agreed, i think all of this falls out a whole lot more simply with interfaces and schemas rather then dictating implementation items like __getattr__ hooks and proxies. ie define the core schema for mailman operations on a list and others can extend as needed for their custom attributes and serialization. as you say, newstyle class properties offer the flexibility of attribute access with arbitrary impl. additionally it would be nicer if we can break down the core schema to its component parts, ie mailman list transport schema vs. mailman list web schema, or use adaptation where possible. i'm curious what got done at the sprint, is there a summary available anywhere? cheers, -kapil From rmunn at pobox.com Wed Mar 30 03:14:54 2005 From: rmunn at pobox.com (Robin Munn) Date: Wed Mar 30 03:15:00 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <2FEA2B52-A0AC-11D9-B064-000393DB3C38@objectrealms.net> References: <4248ECBC.8080509@pobox.com> <42496D27.20104@gaiahost.coop> <16736.1112138883@kanga.nu> <2FEA2B52-A0AC-11D9-B064-000393DB3C38@objectrealms.net> Message-ID: <4249FD8E.7060800@pobox.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Kapil Thangavelu wrote: | i'm curious what got done at the sprint, is there a summary available | anywhere? | | cheers, | | -kapil I'll see if I can write up a summary and put it on the wiki. The one-paragraph version: we got the core database schema implemented (using SQLObjects) and some unit tests, and the beginnings of some business objects (the MailingList object in src/mailman/mlist.py). Robin Munn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCSf2K6OLMk9ZJcBQRAnJ9AJ0aTL6wBGdG4b5b2LAiDhun4gArEACfR3kx g23lyogt/o91rzSAxRwiab4= =i9N7 -----END PGP SIGNATURE----- From barry at python.org Wed Mar 30 05:15:46 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 30 05:15:49 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <4248ECBC.8080509@pobox.com> References: <4248ECBC.8080509@pobox.com> Message-ID: <1112152545.30032.321.camel@presto.wooz.org> On Tue, 2005-03-29 at 00:50, Robin Munn wrote: > During the PyCon sprints, we created a MailingList object in > src/mailman/mlist.py which has __getattr__() and __setattr__() methods > to set mailing list attributes such as "max_hold" or "enable_archives". > I've been rethinking that decision; I think another method might be > slightly better. Instead of using straight attribute access, I think it > might be better to use a proxy object. Call it "attr", or maybe just > "a". Possibly both could be allowable. That's a good point, and something I'd thought about during the first rewrite (I'm not sure that made it into the branches snapshot though). I too dislike having to type that extra dotted-path for /every/ attribute access, but it may well be something we have to add. OTOH, maybe we don't need to worry much about collisions between attributes of the MailingList class and dynamic, schema-fed attributes. Attribute lookup always returns an entry from the instance's dictionary (or its class's) before calling __getattr__(). For our own dynamic attributes, we need only be internally careful about name collisions. Third party add-ons will have to be doubly careful because first, they can't collide with an attribute we're storing in the database, and second, they can't collide with one of the items in the MailingList's dictionary. So they already have to be careful. I'd be inclined to leave things where they are for now. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050329/500b1ceb/attachment.pgp From barry at python.org Wed Mar 30 05:17:03 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 30 05:17:06 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <42496D27.20104@gaiahost.coop> References: <4248ECBC.8080509@pobox.com> <42496D27.20104@gaiahost.coop> Message-ID: <1112152623.30032.324.camel@presto.wooz.org> On Tue, 2005-03-29 at 09:58, Mark Bucciarelli wrote: > By the way, how did you guys do on Tuesday? I lost a good part of the day with some back spasms, but we started writing some test cases for the MailingList class. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050329/3623ad72/attachment.pgp From barry at python.org Wed Mar 30 05:25:25 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 30 05:25:30 2005 Subject: [Mailman3-dev] Mailing list attributes: immediate access, or proxied? In-Reply-To: <2FEA2B52-A0AC-11D9-B064-000393DB3C38@objectrealms.net> References: <4248ECBC.8080509@pobox.com> <42496D27.20104@gaiahost.coop> <16736.1112138883@kanga.nu> <2FEA2B52-A0AC-11D9-B064-000393DB3C38@objectrealms.net> Message-ID: <1112153125.30038.332.camel@presto.wooz.org> On Tue, 2005-03-29 at 18:42, Kapil Thangavelu wrote: > agreed, i think all of this falls out a whole lot more simply with > interfaces and schemas rather then dictating implementation items like > __getattr__ hooks and proxies. Agreed. My own feeling is that we'll build up that machinery after we get more of the system fleshed out. > ie define the core schema for mailman operations on a list and others > can extend as needed for their custom attributes and serialization. > as you say, newstyle class properties offer the flexibility of > attribute access with arbitrary impl. additionally it would be nicer if > we can break down the core schema to its component parts, ie mailman > list transport schema vs. mailman list web schema, or use adaptation > where possible. The application-level MailingList class's primary purpose in life is to support the attributes and operations that core Mailman needs. At this stage, I think simplicity is our best friend. > i'm curious what got done at the sprint, is there a summary available > anywhere? Not yet, but I'll try to summarize here. I was hoping we'd get a new wiki up soon, but in the meantime, this list will have to do. Our biggest accomplishment was designing the database schema to support as much of the requirements as we currently understand. We tried to do this last year, but I think we got bogged down in the SQL-weeds. This year, things went much more smoothly because we adopted SQLObjects as our interface to the database. The results are in Subversion, and I'm pretty happy with them (modulo an issue I'll bring up in a separate thread). We also wrote a bunch of test cases. So I think we have a good basis from which to work now. I also hope to soon talk about how we can structure this project so that we get much more community participation. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050329/ec349cfb/attachment.pgp From barry at python.org Wed Mar 30 05:33:14 2005 From: barry at python.org (Barry Warsaw) Date: Wed Mar 30 05:33:20 2005 Subject: [Mailman3-dev] Problem with the schema Message-ID: <1112153594.30039.341.camel@presto.wooz.org> I was working on a method to delete mailing lists and I realized that our schema as currently designed doesn't support it. Think about Rosters. In order to support composition of mailing lists, we let Rosters be shared. So for example, it's quite easy in our current schema for List-A to have its membership in one Roster, List-B to have its membership in a second Roster, and List-C to be composed of Roster-A and Roster-B. But what happens if List-B gets deleted? Do we delete Roster-B? Do we leave it hanging around because List-C refers to it? Also, can we know how many mailing lists are referring to a particular Roster? Remember too that we can re-use Rosters as the list of owners or moderators. Similar issues arise through the re-use of LanguageTextCollections and MessageCollections. Essentially, we'd need to add reference counting to these tables (or implement the equivalent through complicated queries). The question is whether it makes sense to "delete a mailing list", especially when we have to support the use case of deleting a list but keeping its archives around for posterity. So what happens to its MessageCollection pointed to by the 'archives' column? Maybe we should just be marking a MailingList as disabled. If all we care about is a row in the DBMailingList table (yes, I've renamed them ;), then maybe it's cheap enough to keep the row around and just set a flag saying whether the list is active or not. Anyway, I'm not sure what we should do, so suggestions are welcome! -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 307 bytes Desc: This is a digitally signed message part Url : http://mail.python.org/pipermail/mailman3-dev/attachments/20050329/9cdeb425/attachment-0001.pgp From claw at kanga.nu Wed Mar 30 06:12:49 2005 From: claw at kanga.nu (J C Lawrence) Date: Wed Mar 30 06:12:53 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: Message from Barry Warsaw of "Tue, 29 Mar 2005 22:33:14 EST." <1112153594.30039.341.camel@presto.wooz.org> References: <1112153594.30039.341.camel@presto.wooz.org> Message-ID: <23704.1112155969@kanga.nu> On Tue, 29 Mar 2005 22:33:14 -0500 Barry Warsaw wrote: > Think about Rosters. In order to support composition of mailing > lists, we let Rosters be shared. So for example, it's quite easy in > our current schema for List-A to have its membership in one Roster, > List-B to have its membership in a second Roster, and List-C to be > composed of Roster-A and Roster-B. Yup. Conclusion: Lists and Rosters are first class objects. > But what happens if List-B gets deleted? Do we delete Roster-B? No. Rosters as first class objects have a set of expiry semantics which are unique to themselves. Those semantics may be the super-set of the expiry semantics of the lists which reference them, or some other arbitrary set (which may cause them to be expired before their dependent lists expire). > Do we leave it hanging around because List-C refers to it? You leave it hanging around until it, itself, decides to die. > Also, can we know how many mailing lists are referring to a particular > Roster? Yuo can handle that via a periodic mark-sweep pass as the number of atoms involved is generally not large. > Remember too that we can re-use Rosters as the list of owners or > moderators. Yup, that's why they have their own survival semantics. Also consider the case of a list which posesses multiple rosters, only some of which are r/w. > Similar issues arise through the re-use of LanguageTextCollections and > MessageCollections. I'd treat MessageCollections as another first class object with discrete semantics. I don't know what LanguageTextCollections are yet. > The question is whether it makes sense to "delete a mailing list"... There are multiple levels of "deletion": - Doesn't accept or send mail any more - List is invisible to external view but still exists as a set of meta-data configurations. - Doesn't exist as a list configuration but still has archives and other meta-data - No trace remains. - etc. As with all these things I'd suggest starting with determining what the minimum necessary atoms are, what the necessary relationships among atoms are, and then building from there. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw@kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live. From mark at gaiahost.coop Wed Mar 30 15:29:22 2005 From: mark at gaiahost.coop (Mark Bucciarelli) Date: Wed Mar 30 15:28:31 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: <1112153594.30039.341.camel@presto.wooz.org> References: <1112153594.30039.341.camel@presto.wooz.org> Message-ID: <424AA9B2.6000203@gaiahost.coop> Barry Warsaw wrote: > But what happens if List-B gets deleted? Do we delete Roster-B? Do we > leave it hanging around because List-C refers to it? I guess it makes sense to leave Rosters hanging around as a separate entity. Perhaps a checkbox (default to True) on the mailing list delete page that says "Remove orphan Rosters?" Although every config option increases complexity and the sample space for tests. But I'm not sure what the most common use case would be. > Also, can we know > how many mailing lists are referring to a particular Roster? Remember > too that we can re-use Rosters as the list of owners or moderators. I don't have the schema.py handy on this box (and I can't remember the svn url--list.org/svn/mailman3 didn't work), but somewhere we must store the link between the list(s) and roster(s). > The question is whether it makes sense to "delete a mailing list", > especially when we have to support the use case of deleting a list but > keeping its archives around for posterity. For me, it would violate the principle of least astonishment if I could not vaporize a list and everything associated with it. I think JCL's suggestion was good--there are different levels of delete. Regards, Mark P.S. No need to copy me on replies, I am subscribed. From iane at sussex.ac.uk Wed Mar 30 17:08:09 2005 From: iane at sussex.ac.uk (Ian Eiloart) Date: Wed Mar 30 17:08:11 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: <1112153594.30039.341.camel@presto.wooz.org> References: <1112153594.30039.341.camel@presto.wooz.org> Message-ID: --On March 29, 2005 22:33:14 -0500 Barry Warsaw wrote: > I was working on a method to delete mailing lists and I realized that > our schema as currently designed doesn't support it. > > Think about Rosters. In order to support composition of mailing lists, . . . > > Anyway, I'm not sure what we should do, so suggestions are welcome! > -Barry > Hi, Sounds to me like you're thinking that a roster is a property of a mailing list. That must be incorrect, it's the situation that you want to get away from. At our site, we're likely to have more rosters than mailing lists. If you're using a relational database for this stuff, then the rosters will belong in one table: ROSTERS = rostername | userid | etc... lists will belong in another: LISTS = listname | listdomain | archived | disabled | etc... Another table will have ACLs for rosters: ACLS = rostername | listname | listdomain | owner | moderator | recipient | poster | etc... Now, it may be that you want to have some special rosters that exist ONLY for specific lists. For example, when creating a list FOO in domain BAR, you might want to create a roster FOO@BAR_recipients for the default recipient list. you might also want _owners, _moderators, _poster etc. I'd also want an _exceptions: for people who can't get off a roster like "staff", but want off of a particular list. Removing orphan rosters would only be desirable if they belonged to that set of special rosters. While I'm here, it would be nice to have pseudo-users for "posters", with wildcards like *@example.com to allow all local users to post to a list. This scheme would allow you to delete lists without worrying about losing rosters, but maintain 2.x functionality for those that don't care about rosters. You'd delete a list by deleting it's one entry in the LISTS table, all related entries in the ACLS table, and all the special rosters. You could disable a list by flagging it as disabled, or (better) setting a disabled date in the past. Most admins would want to disable a list for a while before deleting it. -- Ian Eiloart Servers Team Sussex University ITS From jpr at uab.edu Wed Mar 30 18:58:34 2005 From: jpr at uab.edu (John-Paul Robinson) Date: Wed Mar 30 18:59:12 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: Message-ID: Agreed. It sounds like a roster is list subscriber namespace construction mechanism. When viewed from the perspective of a specific list, the list name is the root of the tree, the rosters are internal nodes, and the subscribers are the leaf nodes. It seems that a list "delete" would relate to the deconstruction of this heirarchical relationship and not necessarily the destruction of its constituent parts. Until a roster is orphaned (no longer has any references) it is not eligable for deletion (similar to a hard link in a traditional file system). Note, in cases where the subscribers and rosters come from an external source, the "zero reference" condition may not be under the control of mailman. That is, a roster may always have > 0 reference in this situation and mailman would never try to delete it. An example of what I'm thinking of: I construct a mailing list of all students in a department based on the rosters from all offered courses in the department. This data may be external to mailman and mailman has no responsiblility for managing it. I would be interested in telling mailman to construct a list named cs-students using rosters for cs101, cs201, cs301, etc. ~jpr On Wed, 30 Mar 2005, Ian Eiloart wrote: > > > --On March 29, 2005 22:33:14 -0500 Barry Warsaw wrote: > > > I was working on a method to delete mailing lists and I realized that > > our schema as currently designed doesn't support it. > > > > Think about Rosters. In order to support composition of mailing lists, > . > . > . > > > > Anyway, I'm not sure what we should do, so suggestions are welcome! > > -Barry > > > > Hi, > > Sounds to me like you're thinking that a roster is a property of a mailing > list. That must be incorrect, it's the situation that you want to get away > from. > > At our site, we're likely to have more rosters than mailing lists. If > you're using a relational database for this stuff, then the rosters will > belong in one table: > > ROSTERS = rostername | userid | etc... > > lists will belong in another: > > LISTS = listname | listdomain | archived | disabled | etc... > > Another table will have ACLs for rosters: > > ACLS = rostername | listname | listdomain | owner | moderator | recipient | > poster | etc... > > > Now, it may be that you want to have some special rosters that exist ONLY > for specific lists. For example, when creating a list FOO in domain BAR, > you might want to create a roster FOO@BAR_recipients for the default > recipient list. you might also want _owners, _moderators, _poster etc. I'd > also want an _exceptions: for people who can't get off a roster like > "staff", but want off of a particular list. > > Removing orphan rosters would only be desirable if they belonged to that > set of special rosters. > > While I'm here, it would be nice to have pseudo-users for "posters", with > wildcards like *@example.com to allow all local users to post to a list. > > This scheme would allow you to delete lists without worrying about losing > rosters, but maintain 2.x functionality for those that don't care about > rosters. You'd delete a list by deleting it's one entry in the LISTS table, > all related entries in the ACLS table, and all the special rosters. You > could disable a list by flagging it as disabled, or (better) setting a > disabled date in the past. Most admins would want to disable a list for a > while before deleting it. > > > From claw at kanga.nu Wed Mar 30 23:25:07 2005 From: claw at kanga.nu (J C Lawrence) Date: Wed Mar 30 23:25:19 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: Message from John-Paul Robinson of "Wed, 30 Mar 2005 10:58:34 CST." References: Message-ID: <6944.1112217907@kanga.nu> On Wed, 30 Mar 2005 10:58:34 -0600 (CST) John-Paul Robinson wrote: > Until a roster is orphaned (no longer has any references) it is not > eligable for deletion (similar to a hard link in a traditional file > system). True, and there are other interesting cases. Consider: I have a company. The company is organised into project and focus teams. Each team is represented by a roster which contains all members of that team. A given individual may be part of multiple teams, and thus may be present on multiple rosters. This is especially true for managers and executives. Every project is represented by a mailing list. As projects develop over time they intersect and involve different teams. To represent this rosters are simply andded to and removed from the relevant mailing lists. A given roster may be linked to multiple lists. A given roster may be linked to no lists and remain so unlinked for a considerable period. Such a roster must not be deleted. A given email address may occur multiple tiles across the different rosters that are referenced by a given list. (This is mentioned simply because only one email should be sent). Rosters are first class objects that require their own distinct handling. > Note, in cases where the subscribers and rosters come from an external > source, the "zero reference" condition may not be under the control of > mailman. That is, a roster may always have > 0 reference in this > situation and mailman would never try to delete it. I'd argue that Mailman should never try to delete a roster unless that roster-type's own expiry polocy states that it should be deleted. The nice this is that this need not be done automatically and instantly. It is a housekeeping function and so can be done cheaply via a low frequency (monthly?) cronjob. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw@kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live. From iane at sussex.ac.uk Thu Mar 31 11:40:46 2005 From: iane at sussex.ac.uk (Ian Eiloart) Date: Thu Mar 31 11:40:47 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: References: Message-ID: <7B219FC667BB64C033B873CA@lewes.staff.uscs.susx.ac.uk> --On March 30, 2005 10:58:34 -0600 John-Paul Robinson wrote: > Agreed. It sounds like a roster is list subscriber namespace > construction mechanism. When viewed from the perspective of a specific > list, the list name is the root of the tree, the rosters are internal > nodes, and the subscribers are the leaf nodes. Alternatively, viewed from the perspective of a subscriber, rosters are internal nodes and lists are leaf nodes. This isn't really a tree - it's a net. -- Ian Eiloart Servers Team Sussex University ITS From claw at kanga.nu Thu Mar 31 12:05:36 2005 From: claw at kanga.nu (J C Lawrence) Date: Thu Mar 31 12:05:41 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: Message from Ian Eiloart of "Thu, 31 Mar 2005 10:40:46 +0100." <7B219FC667BB64C033B873CA@lewes.staff.uscs.susx.ac.uk> References: <7B219FC667BB64C033B873CA@lewes.staff.uscs.susx.ac.uk> Message-ID: <18483.1112263536@kanga.nu> On Thu, 31 Mar 2005 10:40:46 +0100 Ian Eiloart wrote: > Alternatively, viewed from the perspective of a subscriber, rosters > are internal nodes and lists are leaf nodes. This isn't really a tree > - it's a net. It becomes more complex when/if you want to support the concept of a user having an account with the system, to which are registered some number of email addresses, various connections from both the account and individual addresses to rosters and lists, with various arbitrary mail policies attached to each address as well as the account to control/hint mail flow (eg "For rosters Q, R and S, only send messages shorter than NNN to this address"). Yeah, Mailman doesn't want to head there today, but that's where MLMs are heading, and fast. -- J C Lawrence ---------(*) Satan, oscillate my metallic sonatas. claw@kanga.nu He lived as a devil, eh? http://www.kanga.nu/~claw/ Evil is a name of a foeman, as I live. From iane at sussex.ac.uk Thu Mar 31 13:21:53 2005 From: iane at sussex.ac.uk (Ian Eiloart) Date: Thu Mar 31 13:21:55 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: <18483.1112263536@kanga.nu> References: <7B219FC667BB64C033B873CA@lewes.staff.uscs.susx.ac.uk> <18483.1112263536@kanga.nu> Message-ID: --On March 31, 2005 02:05:36 -0800 J C Lawrence wrote: > On Thu, 31 Mar 2005 10:40:46 +0100 > Ian Eiloart wrote: > >> Alternatively, viewed from the perspective of a subscriber, rosters >> are internal nodes and lists are leaf nodes. This isn't really a tree >> - it's a net. > > It becomes more complex when/if you want to support the concept of a > user having an account with the system, Yes, I do want to do that. In fact I want two types of account: local and non-local. A local account is one that is already defined on my LDAP (or whatever) servers - they're students and staff on my campus. A non-local account type would be closer to what Mailman has right now - it could use any non-local mail addresses. The distinction is important for several reasons: 1. I know who local people are, and I can take action against them if they misuse lists. Therefore, I can allow them greater permissions. My instinct would be to allow local accounts to post to ANY of my lists, at least by default. I'd want to be able to revoke that for some lists, and for some individuals. 2. Local users shouldn't have to go through the same sign-up process. I already know their email address, and their authentication credentials. I might want to have them agree to some terms and conditions before making their first post, though. 3. A site should be able to define which are local addresses (a list of regular expressions), and decide whether Local accounts can register non-local addresses. An additional complication: here a person may have one account with several mail aliases (I'm iane AND i.eiloart, for example). But they may also have several accounts. Management of people with several accounts may be beyond the scope of Mailman 3 - but it may be that there are design decisions (name spaces, for example) that could be influenced by the need to allow for this in the future. And, when it comes to addresses... We manage many mail domains here. Some are synonymous (susx.ac.uk and sussex.ac.uk, for example), and others aren't. Mailman should permit me to define synonymous local mail domains so that when iane@susx.ac.uk is allowed to post to a list, then iane@sussex.ac.uk is too. > to which are registered some > number of email addresses, various connections from both the account and > individual addresses to rosters and lists, with various arbitrary mail > policies attached to each address as well as the account to control/hint > mail flow (eg "For rosters Q, R and S, only send messages shorter than > NNN to this address"). Yeah, Mailman doesn't want to head there today, > but that's where MLMs are heading, and fast. -- Ian Eiloart Servers Team Sussex University ITS From jpr at uab.edu Thu Mar 31 19:20:35 2005 From: jpr at uab.edu (John-Paul Robinson) Date: Thu Mar 31 19:20:39 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: <18483.1112263536@kanga.nu> Message-ID: On Thu, 31 Mar 2005, J C Lawrence wrote: > On Thu, 31 Mar 2005 10:40:46 +0100 > Ian Eiloart wrote: > > > Alternatively, viewed from the perspective of a subscriber, rosters > > are internal nodes and lists are leaf nodes. This isn't really a tree > > - it's a net. Yes, it is a net under the hood (when all elements are considered), but the view from any particular point can be a tree. So from the perpective of the subscriber its a tree stemming from the subscriber address, with rosters as internal nodes and lists as leaves. > It becomes more complex when/if you want to support the concept of a > user having an account with the system, to which are registered some > number of email addresses, various connections from both the account and > individual addresses to rosters and lists, with various arbitrary mail > policies attached to each address as well as the account to control/hint > mail flow (eg "For rosters Q, R and S, only send messages shorter than > NNN to this address"). Yeah, Mailman doesn't want to head there today, > but that's where MLMs are heading, and fast. I wouldn't say it becomes even more complex since the individual addresses are now rooted under a common node (the account). This is just adding a parent level to the subscriber view tree. The point is that while the objects (lists, rosters, addresses, accounts) are a mesh, the specific relationships can be trees. The subscription and list creation actions relate to defining the relationship links and not reorganizing the objects into new containers. If the implementation can handle the list-roster-address relationships it should be able to add accounts in a straight forward manner. The options you mention seem to be attributes of accounts, addresses, rosters, and lists. Owners, admins, users can set the attributes and then as the message "moves" through the delivery net (list->roster->address->account) they act as filters to cull non-complient messages. Then there are some lists which just apply the list-level attribute filters and simply rely on the (list->roster->address) path for delivery address expansion. That is, you will get the message as long as it meets the list criteria. Yuck. That could get ugly. Seems like it may be best to just have (account,address,list) tuples for attribute setting and matching. Just thinking this through out loud. ~jpr From donn at u.washington.edu Thu Mar 31 20:12:11 2005 From: donn at u.washington.edu (Donn Cave) Date: Thu Mar 31 22:36:48 2005 Subject: [Mailman3-dev] Problem with the schema In-Reply-To: Message-ID: <66E67302-A210-11D9-B3A7-000A959133A8@u.washington.edu> On Thursday, March 31, 2005, at 03:21 AM, Ian Eiloart wrote: > --On March 31, 2005 02:05:36 -0800 J C Lawrence wrote: > >> On Thu, 31 Mar 2005 10:40:46 +0100 >> Ian Eiloart wrote: >> >>> Alternatively, viewed from the perspective of a subscriber, rosters >>> are internal nodes and lists are leaf nodes. This isn't really a tree >>> - it's a net. >> >> It becomes more complex when/if you want to support the concept of a >> user having an account with the system, > > Yes, I do want to do that. In fact I want two types of account: local > and non-local. A local account is one that is already defined on my > LDAP (or whatever) servers - they're students and staff on my campus. > A non-local account type would be closer to what Mailman has right now > - it could use any non-local mail addresses. For what it's worth, that's exactly what we do here (University of Washington central site.) This morning I count 9308 lists, 115676 site local list members subscribed under 117610 email addresses. Couldn't say without a lot more work how many external members. > The distinction is important for several reasons: > > 1. I know who local people are, and I can take action against them > if they misuse lists. Therefore, I can allow them greater permissions. > My instinct would be to allow local accounts to post to ANY of my > lists, at least by default. I'd want to be able to revoke that for > some lists, and for some individuals. Only site local members can administer a list, for this reason. Posting is governed by list membership as usual, though. I think this is the list admin's call, not the site admin's, and Mailman already supports that pretty well out of the box. > 2. Local users shouldn't have to go through the same sign-up > process. I already know their email address, and their authentication > credentials. I might want to have them agree to some terms and > conditions before making their first post, though. Check. For on site users, HTTP access can use web "single sign-on", which is better for several reasons and allows us to skip the confirm ritual for subscription. > 3. A site should be able to define which are local addresses (a > list of regular expressions), and decide whether Local accounts can > register non-local addresses. We use a database that maps site local identifiers (login names) to the email address as used to sign up. Often the latter is closely related to the former, but by no means always. > An additional complication: here a person may have one account with > several mail aliases (I'm iane AND i.eiloart, for example). But they > may also have several accounts. Management of people with several > accounts may be beyond the scope of Mailman 3 - but it may be that > there are design decisions (name spaces, for example) that could be > influenced by the need to allow for this in the future. You're iane and i.eiloart outside of Mailman? I think that would not be Mailman's problem. > And, when it comes to addresses... We manage many mail domains here. > Some are synonymous (susx.ac.uk and sussex.ac.uk, for example), and > others aren't. Mailman should permit me to define synonymous local > mail domains so that when iane@susx.ac.uk is allowed to post to a > list, then iane@sussex.ac.uk is too. Yeah, this is one we haven't tackled. Anyway, this obviously took quite a bit of changes to Mailman, but it was certainly worth it for our site, I can't imagine trying to support the out-of-the-box model here. The code is available via http://staff.washington.edu/donn/UWmm.html We have had some inquiries, though I can't say for sure anyone else has actually deployed it. It isn't trivial to apply these changes and make them work at your own site. Donn Cave, donn@u.washington.edu