From syst3m.w0rm at gmail.com Fri Jun 1 00:22:12 2012 From: syst3m.w0rm at gmail.com (Aamir Khan) Date: Thu, 31 May 2012 18:22:12 -0400 Subject: [Mailman-Developers] HyperKitty login In-Reply-To: <4FC7DB80.50406@state-of-mind.de> References: <87y5oathb9.fsf@uwakimon.sk.tsukuba.ac.jp> <1373C8EA-7380-4281-AD70-5669CF43849E@NFSNet.org> <8762bc3hlb.fsf@uwakimon.sk.tsukuba.ac.jp> <4FC7DB80.50406@state-of-mind.de> Message-ID: I am sorry for not drafting my email properly. "social_auth" is already being used by postorius so I thought that you guys might be aware of it. But, I was wrong..and I will keep this in mind for future mails to the list. Regarding blog posts, I have absolutely no problem in writing blog posts about my progress but I don't think that writing blog posts at such an early stage is going to help in any way. Once, I will make a significant progress I will blog about it. I think most of the questions raised are already covered. I will be more than happy to answer any further questions you might have. -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee From pingou at pingoured.fr Fri Jun 1 15:03:23 2012 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Fri, 01 Jun 2012 15:03:23 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120531172852.GO2569@unaka.lan> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <1335458162.10650.0.camel@ambre.pingoured.fr> <20120531172852.GO2569@unaka.lan> Message-ID: <1338555803.21165.15.camel@ambre.pingoured.fr> On Thu, 2012-05-31 at 10:28 -0700, Toshio Kuratomi wrote: > > The idea is that now, we can have different backend and each module > > needing access to the emails can use the API directly without having > to > > bother about which storage system is behind. > > > Wacky looked at this today and asked if we should have the > x-message-id-hash as another key value to look up an email. I am all for extending the API as much as we need, and I am sure the different modules have different needs, so please let us know what you would like to have access to. Pierre From barry at list.org Sat Jun 2 03:02:55 2012 From: barry at list.org (Barry Warsaw) Date: Fri, 1 Jun 2012 21:02:55 -0400 Subject: [Mailman-Developers] HyperKitty MongoDB vs PostgreSQL. In-Reply-To: <20120525155901.GG2569@unaka.lan> References: <1337937537.8091.19.camel@ambre.pingoured.fr> <20120525110310.478c8b41@limelight.wooz.org> <20120525155901.GG2569@unaka.lan> Message-ID: <20120601210255.0497c8a2@resist.wooz.org> On May 25, 2012, at 08:59 AM, Toshio Kuratomi wrote: >We're using SQLAlchemy i(a different ORM) to abstract the database layer. At one point the core used SA, but this was many years ago and I had lots of problems with it. IIRC it was mostly, when things went south, it was quite difficult to debug. I'm sure things have improved since then (and SA is Python 3 compatible, whereas Storm is not, so... :). >That means the big three open source solutions (sqlite, postgres, and mysql) >and proprietary ones like oracle should all be supported. However, we're >thinking about using fulltext search indexes which would mean tying it to >specific backends. It may be possible to run in a degraded mode (searches >being slower... but for small to medium lists this is likely fine) so we >could keep the backend abstraction while running faster on certain >databases.. That's totally reasonable, and mirrors our policy on JavaScript. >MongoDB is not a relational database so using an ORM seems like a bit of >a hack. People have adapted SQLAlchemy for use with MongoDB but I'm not >sure what sacrifices they had to make to make that work. Agreed. >Running comparisons using postgres was actually a choice dictated by our >experience with the Core. In our use, using the sqlite backend quickly >resulted in database locks that we couldn't figure our way out of. So that >effectively left us with postgres as the only db for the core. If admins >have to run postgres for the Core then the thinking is that it wouldn't be >hard for them to also run it for the archiver. I've never had the sqlite >issue with SQLAlchemy,though, so we could probably support >a non-fulltext-optimized search in sqlite without doing much of anything. This is a serious question, and I know that multiproc sqlite applications are prone to database lockups. They're also notoriously difficult to debug, or even figure out enough to file a reasonable bug. I think in general it's because there some path through the code of one of the processes that doesn't end in a commit or abort, and I know I've fixed some of those. It would be sad if a smallish Mailman system couldn't be effectively run on SQLite, just because it's so much easier to set up. Certainly it's much better for day-to-day development. I guess we'll find out as more real deployments happen. In any case, I'd still like to officially support PostgreSQL, and for that matter, MySQL (and derivatives), but I admit to not testing very often on the former, and no one has contributed support for the latter yet. We really need some buildbots. :) -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Sat Jun 2 03:08:13 2012 From: barry at list.org (Barry Warsaw) Date: Fri, 1 Jun 2012 21:08:13 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> Message-ID: <20120601210813.0e4fc625@resist.wooz.org> Jeebus, finally catching up on this thread. On Apr 24, 2012, at 01:41 PM, Stephen J. Turnbull wrote: >IMHO, premature optimization. Among other things, there isn't going >to be a "the" archiver-core. Mailman should provide "a" >archiver-core, and I think it should be based on maildir (which is >apparently Barry's intuition, too). Theory and implementation of >maildir are simple and robust, and that allows us to concentrate on >the archiver interface. In fact, the prototype archiver already implements maildir storage. This storage isn't exposed via REST yet though. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Sat Jun 2 03:33:50 2012 From: barry at list.org (Barry Warsaw) Date: Fri, 1 Jun 2012 21:33:50 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120424181221.GM28774@unaka.lan> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> Message-ID: <20120601213350.10e50c01@resist.wooz.org> On Apr 24, 2012, at 11:12 AM, Toshio Kuratomi wrote: >Ive been thinking about this and I'm in mild disagreement. I think that >a mailing list system should give people an archive-store which is acessible >behind a generalized API. I'm warming up to this. The IArchiver interface is generic enough to support both internal and external archivers. If there are deficiencies in either, we can fix the API, as long as both use cases are supportable, in a manner similar to IArchiver.permalink() returning None if the archiver doesn't support stable urls. (A known omission from the current IArchiver API is that there's no way to access attachments. Does anybody have good ideas about that?) >The store may be accessible via a REST API but I'm not certain that its the >correct level to deal with when talking about it in this contect. The >current mailman3 doesn't have an API for plugging in archivers via REST... >it has an API for plugging in archivers via python. That may be the correct >level to be looking at this. From a systems perspective, yes. Archivers must be enabled system-wide via the config file, but I think we should allow individual lists to opt-in or -out of system-enabled archivers. I'm on the fence as to what to do about the prototype archiver, which is beginning to seem much more like the default archiver-core, i.e. sans ui. >I think we should look into something a little more symmetrical: > >[mailman3 core] -- maintainance of list metadata, sending and receiving, > provides a REST API > [Web UIs] -- web ui to Core functions > [Archive-stores] -- stores the messages sent to the mailing lists. > Provides a (REST?) API to apps built on top of it > [Archiver UIs] -- web ui, nntp interface, REST API (if not implemented > at the storage layer), etc to the archive-store This is compelling. >Question: Why have multiple stores? The big reason is that archives are >being much more rapidly developed right now. So I anticipate that people >are going to be working on different storage technology with different >tradeoffs. One storage might be faster. Another might be more generally >available. We'll have to reexamine this in the future. It's possible that >we'll find one storage system that is perfect for all cases. It's also >possible that we'll find all storage solutions have tradeoffs in which case >we'll likely want to support third-party stores forever. I always envisioned the core's storage being splittable into three main partitions. One would be the list-centric data, another would be the user-centric data, and the third would be the message-centric data. If you look carefully for example, you'll see that there are no direct foreign key references between members and the mailing lists they're associated with. This link is by fqdn listname, *not* mailinglist table ids. This is deliberate. (It's entirely possible the implementation doesn't actually allow these three partitions to be stored in completely separate places. I'd consider that a bug.) OTOH, I don't think it makes sense for the core to rely on more than one ORM. For now, that's Storm. (I'm slightly lying here because the technology that shows the most promise for supporting schema migrations is Alembic which is based on a stripped down version of SQLAlchemy. But migrations are probably a completely off-line operation.) >Question: This is all dangling off of the archiver interface for mailman3 >anyway so how can we affect the outcome? Well, in some ways people can >create anything they want in there so we cant enforce a solution. However, >if we think that it's desirable, we can certainly document this (maybe with >an interface if we go the python route for that layer of API or with >a specification of what the REST API should look like for that.) We can >also enhance our current archivers to provide the API that we come up with. >I have a feeling that the prototype archiver with maildir will be a little >slow but if it provides the API and comments about separation between >core, storage, and archive UI it gives people a starting point to >creating their own. Some IArchiver implementations will be purely external archivers. I like that we can have a Mail Archive implementation, or potentially a Gmane implementation. Those are very different from a MHonArc implementation, which is again different from the prototype (default? built-in? always-enabled?) archiver. Having a common API for all of these simplifies the parts of the core that send messages to the archives, but what happens once the data is inserted into the different archivers is another question. Remember too that archiver speed is less important, since that doesn't live in the critical path for message delivery. There is a handle that basically copies the message to the archiver queue, and there's a separate runner that dequeues those messages and sends them off to the individual archivers, via the IArchive interface. So I think the performance of message insertion isn't something we should worry about for now. >Question: Where do we start? I think that we'll either succeed or fail very >quickly by trying to define what the API between archive-store and >archiver-ui should look like. We'll either be able to agree on a common set >of features there (from which we'll be able to go forth and create our own >archive-storage plugins) or we'll decide that we all need/want to do >different things that no common API can address. If there's no common API >definition then we won't be able to do any of the rest of this so there >won't be any sense continuing down that path. Places to start: * Look at the IArchiver interface and try to figure out whether it's complete from a message-insertion POV. Maybe in that case, we don't care about attachments since the archiver will do whatever it wants with them. * Look at the IMessageStore API. Is this complete? IOW, could you build a purely Python-level archiver like HyperKitty on top of this API? Here's where proper attachment handling would probably be necessary. * How would you want to expose the IMessageStore interface into the REST API? My sense is that you could probably take a fairly straightforward translation of IMessageStore into REST and *that* would be what you'd build the various archiver UIs on top of. REST needs to answer questions like batching which are necessary for efficient transfer of data over HTTP but not for direct Python calls. * Should threading information be part of the IMessageStore, or a separate interface? If the prototype archiver becomes the default implementation for the IMessageStore, it probably needs to grow a lot more functionality to support threading information. The way I'm seeing it is that IArchiver is the interface for getting messages *into* the IMessageStore. The IMessageStore is the interface for making Python level queries needed to get the raw messages out of the system, and a REST API is how you publish this data for the various ui consumers. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Sat Jun 2 03:38:21 2012 From: barry at list.org (Barry Warsaw) Date: Fri, 1 Jun 2012 21:38:21 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <1335458162.10650.0.camel@ambre.pingoured.fr> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <1335458162.10650.0.camel@ambre.pingoured.fr> Message-ID: <20120601213821.3083d672@resist.wooz.org> On Apr 26, 2012, at 06:36 PM, Pierre-Yves Chibon wrote: >The current version of HK relies on mongodb for the storage, but I want >to test HK with a traditionnal SQL backend. So I have started to work on >this. > >The interface I defined is there: >https://github.com/pypingou/kittystore/blob/master/kittystore/__init__.py > >And its implementation using SQLAlchemy is there: >https://github.com/pypingou/kittystore/blob/master/kittystore/kittysastore.py > >The mongodb implementation isn't done yet but should be quite trivial to >do (most function from the API were coming from it). > >The idea is that now, we can have different backend and each module >needing access to the emails can use the API directly without having to >bother about which storage system is behind. Follow on thoughts to my previous message. Let's say you hate the default prototype archiver because mbox is too slow. Further, let's say you have an amazing implementation of the backend message store based on mongodb. How does that fit into my previous picture? Actually quite easily I think. As long as you can expose insertion into the archiver with the IArchiver interface, and extraction from the archiver via the IMessageStore interface, these two bits can replace the default implementations, just by changing how the ZCA maps the interfaces to implementations. This mongodb-based message-storage-core can even live outside the core process, and *still* be available to in-process Python code, or exposed by the core in its REST API. It would just take a little extra IPC hidden behind the implementations of those two APIs. Cheers, -Barry From barry at list.org Sat Jun 2 03:39:20 2012 From: barry at list.org (Barry Warsaw) Date: Fri, 1 Jun 2012 21:39:20 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120531172852.GO2569@unaka.lan> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <1335458162.10650.0.camel@ambre.pingoured.fr> <20120531172852.GO2569@unaka.lan> Message-ID: <20120601213920.7cd2ff5c@resist.wooz.org> On May 31, 2012, at 10:28 AM, Toshio Kuratomi wrote: >Wacky looked at this today and asked if we should have the x-message-id-hash >as another key value to look up an email. That seems proper to me. +1 >Would we want a separate function or to overload get_email() so that it can >either take message_id or message_id_hash? They are separate methods in the IMessageStore API. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From p at state-of-mind.de Sat Jun 2 07:19:10 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sat, 2 Jun 2012 07:19:10 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120601210813.0e4fc625@resist.wooz.org> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120601210813.0e4fc625@resist.wooz.org> Message-ID: <20120602051909.GD16571@state-of-mind.de> * Barry Warsaw : > On Apr 24, 2012, at 01:41 PM, Stephen J. Turnbull wrote: > >IMHO, premature optimization. Among other things, there isn't going > >to be a "the" archiver-core. Mailman should provide "a" > >archiver-core, and I think it should be based on maildir (which is > >apparently Barry's intuition, too). Theory and implementation of > >maildir are simple and robust, and that allows us to concentrate on > >the archiver interface. > > In fact, the prototype archiver already implements maildir storage. This > storage isn't exposed via REST yet though. Maildir is robust but it doesn't scale under high load. You can add indexes, but they are limited sooner or later too. Concerning mailbox formats Timo Sirainens current approach to collect a limited number of messages in one file and then start a new one combines the best of both worlds - mbox and maildir - in mdbox . Con It takes an index to know in which files a message is located. Pro A magnitude faster to backup, which I would keep an eye on because mailing list archives tend to be large and backing up a directory of small files is a well known performance killer. I can get you in contact with Timo if you like to. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 316 bytes Desc: Digital signature URL: From pingou at pingoured.fr Sun Jun 3 14:49:55 2012 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Sun, 03 Jun 2012 14:49:55 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120601213350.10e50c01@resist.wooz.org> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <20120601213350.10e50c01@resist.wooz.org> Message-ID: <1338727795.3151.7.camel@ambre.pingoured.fr> On Fri, 2012-06-01 at 21:33 -0400, Barry Warsaw wrote: > * Look at the IMessageStore API. Is this complete? IOW, could you > build a purely Python-level archiver like HyperKitty on top of this > API? Here's where proper attachment handling would probably be > necessary. I have defined a number of functions in the kittystore interface which could go into the IMessageStore: https://github.com/pypingou/kittystore/blob/master/kittystore/__init__.py Main difference compare to what is in the IMessageStore at the moment is that I require the fq list name as first argument of each function (ie: I have one table for each list and need to know where you would like me to search) > * How would you want to expose the IMessageStore interface into the > REST API? > My sense is that you could probably take a fairly straightforward > translation of IMessageStore into REST and *that* would be what you'd > build the various archiver UIs on top of. REST needs to answer > questions like batching which are necessary for efficient transfer of > data over HTTP but not for direct Python calls. +1 for the straightforward translation of the interface to REST. Archiver could then use either the implementation of the interface or REST. > * Should threading information be part of the IMessageStore, or a > separate interface? If the prototype archiver becomes the default > implementation for the IMessageStore, it probably needs to grow a lot > more functionality to support threading information. Keeping some info about threading in the database, the KittyStore interface allows to retrieve for example all the messages of a thread or the length of a thread. I can work on merging KittyStore and IMessageStore and providing the REST interface for it. Should I provide also an implementation ? Using PostgreSQL as backend ? Pierre From pingou at pingoured.fr Sun Jun 3 15:00:12 2012 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Sun, 03 Jun 2012 15:00:12 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <1338727795.3151.7.camel@ambre.pingoured.fr> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <20120601213350.10e50c01@resist.wooz.org> <1338727795.3151.7.camel@ambre.pingoured.fr> Message-ID: <1338728412.3151.9.camel@ambre.pingoured.fr> On Sun, 2012-06-03 at 14:49 +0200, Pierre-Yves Chibon wrote: > Main difference compare to what is in the IMessageStore at the moment > is that I require the fq list name as first argument of each function > (ie: I have one table for each list and need to know where you would > like me > to search) hm, the fact that I would require the full list name will cause a problem regarding the `messages` attributes on the IMessageStore. Pierre From barry at list.org Sun Jun 3 19:24:55 2012 From: barry at list.org (Barry Warsaw) Date: Sun, 3 Jun 2012 13:24:55 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120602051909.GD16571@state-of-mind.de> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120601210813.0e4fc625@resist.wooz.org> <20120602051909.GD16571@state-of-mind.de> Message-ID: <20120603132455.5d61878e@resist.wooz.org> On Jun 02, 2012, at 07:19 AM, Patrick Ben Koetter wrote: >Maildir is robust but it doesn't scale under high load. You can add indexes, >but they are limited sooner or later too. > >Concerning mailbox formats Timo Sirainens current approach to collect a >limited number of messages in one file and then start a new one combines the >best of both worlds - mbox and maildir - in mdbox >. > >Con >It takes an index to know in which files a message is located. > >Pro >A magnitude faster to backup, which I would keep an eye on because mailing >list archives tend to be large and backing up a directory of small files is a >well known performance killer. > >I can get you in contact with Timo if you like to. I've chatted with him a few times (I'm a Dovecot user and fan). Would someone like to take a crack at implementing this format either in, or on top of, the Python stdlib mailbox module: http://docs.python.org/library/mailbox.html I'd much rather use something standard (and maintained by someone else!) than a bunch of custom code specific to Mailman. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From p at state-of-mind.de Sun Jun 3 23:21:06 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Sun, 3 Jun 2012 23:21:06 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120603132455.5d61878e@resist.wooz.org> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120601210813.0e4fc625@resist.wooz.org> <20120602051909.GD16571@state-of-mind.de> <20120603132455.5d61878e@resist.wooz.org> Message-ID: <20120603212106.GF2654@state-of-mind.de> * Barry Warsaw : > On Jun 02, 2012, at 07:19 AM, Patrick Ben Koetter wrote: > > >Maildir is robust but it doesn't scale under high load. You can add indexes, > >but they are limited sooner or later too. > > > >Concerning mailbox formats Timo Sirainens current approach to collect a > >limited number of messages in one file and then start a new one combines the > >best of both worlds - mbox and maildir - in mdbox > >. > > > >Con > >It takes an index to know in which files a message is located. > > > >Pro > >A magnitude faster to backup, which I would keep an eye on because mailing > >list archives tend to be large and backing up a directory of small files is a > >well known performance killer. > > > >I can get you in contact with Timo if you like to. > > I've chatted with him a few times (I'm a Dovecot user and fan). +1 > Would someone like to take a crack at implementing this format either in, or > on top of, the Python stdlib mailbox module: > > http://docs.python.org/library/mailbox.html > > I'd much rather use something standard (and maintained by someone else!) than > a bunch of custom code specific to Mailman. Speaking of standards. I discussed this with Timo in private mail and he suggests not to use mdbox directly. It is still under development and he thinks it should not become a standard format like mbox or maildir. To quote Timo from our conversation: I'd prefer using mdbox via Dovecot itself, either via LMTP or dovecot-lda or maybe by adding some "doveadm save" command. Anything else I think would be problematic. Even using Dovecot's library for accessing mdbox would be problematic in some installations if you didn't also read several settings from dovecot.conf (e.g. lock_method). Note: The "'doveadm save' command" above refers to an idea where dovecot would import to be archived messages from a MM3 mailing list into Dovecot (and whatever format has been definded for the mailbox). Maybe - and to pick up an idea Barry had mentioned to me a long time ago about mailing list management directly from a mail client - we would gain the most if we implemented an LMTP client as archiver (better: archive transport). This would introduce the chance to choose among many LMTP servers and their specific, optimized storage format (Dovecot -> mdbox, Cyrus IMAP -> ?) including their servers various IMAP SEARCH capabilities for searches in archives. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 316 bytes Desc: Digital signature URL: From terri at zone12.com Mon Jun 4 22:50:23 2012 From: terri at zone12.com (Terri Oda) Date: Mon, 04 Jun 2012 14:50:23 -0600 Subject: [Mailman-Developers] Fwd: Mailman GSoC Status report due; code checkin due In-Reply-To: <4FCD0B60.2090206@zone12.com> References: <4FCD0B60.2090206@zone12.com> Message-ID: <4FCD1F8F.6090808@zone12.com> I failed to bcc all of you on this, so here's the message I sent to Aamir, just for the record. -------- Original Message -------- Subject: Mailman GSoC Status report due; code checkin due Date: Mon, 04 Jun 2012 13:24:16 -0600 From: Terri Oda To: Aamir Khan Hi Aamir! How's GSoC going for you thus far? We're now starting week 3 and I've got a couple of things I need from you: (1) I need you to mail a status report to the mailman-developers list saying what you've done so far and what you plan to do next week. Probably just a handful of bullet points. If for some reason you're not comfortable with this, you can email me and I'll summarize for the public list. (2) You're supposed to have some code checked in somewhere by the end of today -- can you send me a link? It's ok if it's not merged yet and you're just publishing a personal branch somewhere, but I'd like to see something. Terri From terri at zone12.com Mon Jun 4 23:55:26 2012 From: terri at zone12.com (Terri Oda) Date: Mon, 04 Jun 2012 15:55:26 -0600 Subject: [Mailman-Developers] Fwd: Mailman GSoC Status report due; code checkin due In-Reply-To: <4FCD1F8F.6090808@zone12.com> References: <4FCD0B60.2090206@zone12.com> <4FCD1F8F.6090808@zone12.com> Message-ID: <4FCD2ECE.10506@zone12.com> On 06/04/2012 02:50 PM, Terri Oda wrote: > I failed to bcc all of you on this, so here's the message I sent to > Aamir, just for the record. Whoops, that was just supposed to go to this year's GSoC mentors. Sorry for the intrusion, the rest of you! Terri From jessy at jessykate.com Mon Jun 4 09:07:33 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Mon, 4 Jun 2012 00:07:33 -0700 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter Message-ID: hello! i am setting up mailman 3.0 on ubuntu 12.04. but getting errors when i try to work with mailman through the python interpreter. i hope this is the correct list to post to. here are the steps i took: - installed mailman 3.0.0.b1 in a fresh virtualenv - install is happy, all tests pass (Total: 383 tests, 0 failures, 0 errors in 3 minutes 27.980 seconds.) - default mailman.cfg file, stored in top level mailman dir (eg, it's a sibling to the bin/ directory) - basic postfix config, as descibed here: http://packages.python.org/mailman/src/mailman/docs/MTA.html#postfix - start mailman with bin/mailman start (no errors to be seen) then i go to follow the example on how to create new mailing lists: http://packages.python.org/mailman/src/mailman/commands/docs/create.html but attempting to import mailman packages fails, so it seems the python packages aren't installed? so i run `python setup.py install` in main mailman directory. after that, the imports work, but i get an error at the point of trying to call `command.process(FakeArgs)` as described on the create page. the error is as follows: Traceback (most recent call last): File "", line 1, in File "/home/jessykate/virtualenvs/mailman/local/lib/python2.7/site-packages/mailman-3.0.0b1-py2.7.egg/mailman/commands/cli_lists.py", line 180, in process else system_preferences.preferred_language.code) File "/home/jessykate/virtualenvs/mailman/local/lib/python2.7/site-packages/mailman-3.0.0b1-py2.7.egg/mailman/core/constants.py", line 53, in preferred_language return getUtility(ILanguageManager)[config.mailman.default_language] File "/home/jessykate/virtualenvs/mailman/local/lib/python2.7/site-packages/zope.component-3.12.1-py2.7.egg/zope/component/_api.py", line 169, in getUtility raise ComponentLookupError(interface, name) zope.interface.interfaces.ComponentLookupError: (, '') FWIW, i have also installed the postorius interface and all seems to work from this approach. i can even verify that the mailing list i created through the postorius interface shows up when i issue `bin/mailman lists`. any idea on how to fix this error? am i missing something? i tried a few other of the instruction pages in the 3.0 docs and all generated errors of some kind. so perhaps i'm just not importing/installing the packages properly? any help greatly appreciated. thanks, -- Jessy http://jessykate.com From mark at msapiro.net Tue Jun 5 07:58:20 2012 From: mark at msapiro.net (Mark Sapiro) Date: Mon, 4 Jun 2012 22:58:20 -0700 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems frompython interpreter In-Reply-To: Message-ID: Jessy Kate Schingler wrote: > >then i go to follow the example on how to create new mailing lists: >http://packages.python.org/mailman/src/mailman/commands/docs/create.html > >but attempting to import mailman packages fails, so it seems the python >packages aren't installed? so i run `python setup.py install` in main >mailman directory. after that, the imports work, but i get an error at the >point of trying to call `command.process(FakeArgs)` as described on the >create page. the error is as follows: All the code in the docs was run as part of the doctest suite when you ran the tests, so it presumably works in your install. I suspect the issue is you just don't have paths and things set right. If you actually want to run Python code, you should run it via Mailman's "bin/mailman shell" (or "bin/mailman withlist" which is the same thing). However, for creating lists and other common tasks, there are subcommands of the bin/mailman command. Run "bin/mailman help" for more info. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From pingou at pingoured.fr Tue Jun 5 08:09:21 2012 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Tue, 05 Jun 2012 08:09:21 +0200 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120601213350.10e50c01@resist.wooz.org> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <20120601213350.10e50c01@resist.wooz.org> Message-ID: <1338876561.12954.2.camel@ambre.pingoured.fr> On Fri, 2012-06-01 at 21:33 -0400, Barry Warsaw wrote: > * Look at the IMessageStore API. Is this complete? IOW, could you > build a purely Python-level archiver like HyperKitty on top of this > API? Here's where proper attachment handling would probably be > necessary. With the help of wacky on #mailman this week-end we came up with this "patch". Attached is both the diff and the file itself. This way we can start discussing :) Pierre -------------- next part -------------- A non-text attachment was scrubbed... Name: message.py.diff Type: text/x-patch Size: 6564 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: messages.py Type: text/x-python Size: 7820 bytes Desc: not available URL: From sophron at latthi.com Tue Jun 5 23:59:56 2012 From: sophron at latthi.com (George Chatzisofroniou) Date: Wed, 6 Jun 2012 00:59:56 +0300 Subject: [Mailman-Developers] [GSoC 2012] Metrics In-Reply-To: <20120520150028.GA4302@otacon> References: <20120505162628.4fed030f@limelight.wooz.org> <20120517210928.GB19285@state-of-mind.de> <87y5oopdyk.fsf@uwakimon.sk.tsukuba.ac.jp> <20120520150028.GA4302@otacon> Message-ID: Hi, Here's my report. As always, you can also read it in my blog [1]. -- The coding period has started. I?ve already learned a lot of stuff! Although, I intended to have more work done by now. That?s why i?m going to speed up in the following days. Here?s what i?ve done so far: - I documented the components of the app. I?ve documented storage of counts, parsing of custom tags and generation of HTML output, basic interfacing with mailman, coalescor (a daemon that cleans up the database) and generator (that generates the metrics from scratch). - I coded the most of my models using polymorphism solutions. - I?ve described the syntax of my custom tags. The syntax is defined by the BNF Language description below: ::= ?{%? ?GRAPH? ?%}? ?{%? ?ENDGRAPH? ?%}? := ?{%? ?TABULATE? ?%}? ?{%? ?ENDTABULATE? ?%}? ::= | ?, ? ::= | ?BY? ::= | ::= ?DAILY? | ?MONTHLY? | ?ANNUAL? | (* Omitted. Default value: DAILY *) ::= ?posts? (* The number of total posts *) | ?threads? (* The number of total threads started *) ::= ?subscribers? (* The number of total subscribers *) ::= ?FOR? | ?FROM? ?TO? | ?SINCE? ::= ?LAST MONTH? | ?LAST DAY? | ?YESTERDAY? | ?TODAY? | ?THIS MONTH? | ?THIS YEAR? | DAYS | MONTHS | YEARS ::= ?YESTERDAY? | ?START OF THE MONTH? | ?START OF THE YEAR? | ::= ?TODAY? | ?NOW? ::= YYYY-MM-DD (* International format defined by ISO (ISO 8601) *) ::= | (* Omitted. Default value: Ommited *) ::= ?WHERE? | ?AND? ::= ?STARTSWITH? | ?IS? | ?ENDSIN? ::= ::= -- [1]: http://sophron.latthi.com/gsoc-mailman/ -- George Chatzisofroniou sophron.latthi.com From barry at list.org Wed Jun 6 04:09:44 2012 From: barry at list.org (Barry Warsaw) Date: Tue, 5 Jun 2012 22:09:44 -0400 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: References: Message-ID: <20120605220944.67b50481@resist.wooz.org> Hi Jessy, You're doing everything right, but there's one conceptual step that you're missing, and which isn't evident from the docs. As Mark says, `mailman shell` is the best way to get a Python interactive interpreter shell to play with the internal Mailman API. Why is this better than just running the virtualenv's `python` interpreter directly? It's because there are a bunch of subsystems that have to be initialized before things will work correctly. E.g., the Zope component architecture (ZCA), the logging subsystem, the configuration subsystem, queues, rules, pipelines, etc. The exception you're getting is because the ZCA hasn't been initialized yet. `mailman shell` ensures that everything is initialized, then it gives you an interpreter prompt. Hope that helps. -Barry From syst3m.w0rm at gmail.com Wed Jun 6 06:43:00 2012 From: syst3m.w0rm at gmail.com (Aamir Khan) Date: Wed, 6 Jun 2012 00:43:00 -0400 Subject: [Mailman-Developers] [HyperKitty] Rating of posts Message-ID: Hello everyone, I am working on HyperKitty archiver project. I am working on feature which will allow the logged-in archiver users to rate emails by upvoting/downvoting. Broadly speaking, there are two kind of emails posted on mailing lists: 1. *Announcement :* Reply is not generally required. 2. *Questions :* Most of the emails posted on mailing are some sort of questions. We want our users to upvote the reply (email) which answered the question accurately. It will be easier for other users of archiver to easily find the best answer (by sorting answers by number of upvotes) of the question without going through all the replies posted on same thread. Rating will be done at email level which means each email can be rated. I think there is no point of having a thread level rating since you can't really compare two different questions. Your thoughts? It's pretty much similar to Q/A site www.quora.com. *Database Schema: *Table structure is very straightforward and simple. Table name : ratings table structure emailid int| userid int| value (+1/-1) where, emailid - references to unique entry storing the actual emails. userid - references to unique entry in user table. value - +1 for upvote and -1 for downvote. In this case each upvote/downvote will be saved as a separate row in database. Your suggestions? -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee From alexander at sulfrian.net Wed Jun 6 12:15:50 2012 From: alexander at sulfrian.net (Alexander Sulfrian) Date: Wed, 06 Jun 2012 12:15:50 +0200 Subject: [Mailman-Developers] URGENT: Google Summer of Code status report and code due In-Reply-To: <4FCD219A.4050203@zone12.com> References: <4FCD219A.4050203@zone12.com> Message-ID: <87aa0g690p.wl%alexander@sulfrian.net> At Mon, 04 Jun 2012 14:59:06 -0600, Terri Oda wrote: > Hi Alex! Hi, > We're starting week 3 of Google Summer of Code, and as far as I know, > none of us have heard from you, and we're getting rather worried. > Midterms come up fast and we will have to fail you if you don't get in > touch soon. sorry to be so quite until now. As you already presumed, I was busy the last weeks at the university. Sadly google summer of code is a bit us centric. Here, the summer holidays starting not till 14th of July. But now it gets a little bit relaxed at university and I will be able to start coding this week. Currently I already created a blog[1] to present the recent development I will do. I also created a personal branch on launchpad (you could find it on[2]), but as I stated earlier, I have not pushed any new commits, but I will start soon this week. I hope you could excuse the late start and I hope that I will be able to create great results. > Terri Alex [1] http://blog.animux.de/category/gsoc2012.html [2] https://code.launchpad.net/~alexander-sd4c/mailman/mailman -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From barry at list.org Thu Jun 7 00:22:47 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 6 Jun 2012 18:22:47 -0400 Subject: [Mailman-Developers] URGENT: Google Summer of Code status report and code due In-Reply-To: <87aa0g690p.wl%alexander@sulfrian.net> References: <4FCD219A.4050203@zone12.com> <87aa0g690p.wl%alexander@sulfrian.net> Message-ID: <20120606182247.67f31bbf@resist.wooz.org> On Jun 06, 2012, at 12:15 PM, Alexander Sulfrian wrote: >Currently I already created a blog[1] to present the recent development I >will do. I also created a personal branch on launchpad (you could find >it on[2]), but as I stated earlier, I have not pushed any new commits, >but I will start soon this week. Thanks for the update Alex. I see that you have not yet committed (or at least pushed) any new revisions to that branch. That's fine, but please do keep in mind that it's much better to propose for merging several small feature branches, rather than one huge branch. It may not make too much difference when you're starting out, but as your code stabilizes and you want to see some of it land on trunk, it becomes critical. Smaller branches are easier to review and merge. Of course, we can help you here with that, if necessary. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From pingou at pingoured.fr Wed Jun 6 21:24:23 2012 From: pingou at pingoured.fr (Pierre-Yves Chibon) Date: Wed, 06 Jun 2012 21:24:23 +0200 Subject: [Mailman-Developers] [HyperKitty] Rating of posts In-Reply-To: References: Message-ID: <1339010663.15279.100.camel@ambre.pingoured.fr> On Wed, 2012-06-06 at 00:43 -0400, Aamir Khan wrote: > > *Database Schema: *Table structure is very straightforward and simple. > > Table name : ratings > > table structure > emailid int| userid int| value (+1/-1) > > where, > emailid - references to unique entry storing the actual emails. > userid - references to unique entry in user table. > value - +1 for upvote and -1 for downvote. > > In this case each upvote/downvote will be saved as a separate row in > database. Your suggestions? I actually would like to ask you to think at a broader level. Could you provide us a database schema on how it would look ? You would have a user table, I assume, the tables for the archives and I would like you to consider in your design ratings but also the category and tags assigned to the emails. You can use Dia, inkscape or http://dbdsgnr.appspot.com/app (from which you can export to an image) to draw the database schema. I have given it some thoughts but I have not found yet a solution that satisfies me, so I would like you to think about it and come up with your design. Thanks, Pierre From barry at list.org Thu Jun 7 01:30:13 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 6 Jun 2012 19:30:13 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <1338876561.12954.2.camel@ambre.pingoured.fr> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <20120601213350.10e50c01@resist.wooz.org> <1338876561.12954.2.camel@ambre.pingoured.fr> Message-ID: <20120606193013.511765f6@resist.wooz.org> On Jun 05, 2012, at 08:09 AM, Pierre-Yves Chibon wrote: >With the help of wacky on #mailman this week-end we came up with this >"patch". Attached is both the diff and the file itself. > >This way we can start discussing :) Point of order: it's much easier to deal with branches and merge proposals (even for works-in-progress) than it is patches in a mailing list thread. :) In any case, a few comments. Why do we need both generic versions of add/delete/get and list-centric versions of those methods? When a message comes into the system, how would we know which to call, or do we call them both? If we decide to keep just the list-centric versions, then it's probably better to take an IMailingList object as the first parameter, and use that to get the fqdn_listname if necessary. Another way of handling search might be to accept keyword arguments, e.g. def search(mlist, **kws) then the keys of kws could be headers, with values being the search term you're looking for. You could define something like _body as the key for searching the body (the entire plain text? one of the attachments?). If we don't need the list-centric versions, using __len__() would be better than get_list_size(). Cheers, -Barry From barry at list.org Thu Jun 7 01:36:03 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 6 Jun 2012 19:36:03 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <20120603212106.GF2654@state-of-mind.de> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120601210813.0e4fc625@resist.wooz.org> <20120602051909.GD16571@state-of-mind.de> <20120603132455.5d61878e@resist.wooz.org> <20120603212106.GF2654@state-of-mind.de> Message-ID: <20120606193603.4c8dd9ad@resist.wooz.org> On Jun 03, 2012, at 11:21 PM, Patrick Ben Koetter wrote: >Maybe - and to pick up an idea Barry had mentioned to me a long time ago about >mailing list management directly from a mail client - we would gain the most >if we implemented an LMTP client as archiver (better: archive transport). > >This would introduce the chance to choose among many LMTP servers and their >specific, optimized storage format (Dovecot -> mdbox, Cyrus IMAP -> ?) >including their servers various IMAP SEARCH capabilities for searches in >archives. I really like the idea of an LMTP client archiver (transport). Does anybody want to take a crack at this? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Thu Jun 7 01:37:46 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 6 Jun 2012 19:37:46 -0400 Subject: [Mailman-Developers] Speaking about kitties (or archivers) In-Reply-To: <1338727795.3151.7.camel@ambre.pingoured.fr> References: <1335205036.2188.29.camel@ambre.pingoured.fr> <20120423182018.50067886@resist.wooz.org> <20120424181221.GM28774@unaka.lan> <20120601213350.10e50c01@resist.wooz.org> <1338727795.3151.7.camel@ambre.pingoured.fr> Message-ID: <20120606193746.686fbf43@resist.wooz.org> On Jun 03, 2012, at 02:49 PM, Pierre-Yves Chibon wrote: >I can work on merging KittyStore and IMessageStore and providing the REST >interface for it. Should I provide also an implementation ? Using PostgreSQL >as backend ? It will be much easier to test with a SQLite backend, although as I'm finding now, SQLite makes it *much* more difficult to migrate the schema. :( -Barry From tanstaafl at libertytrek.org Thu Jun 7 12:43:46 2012 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Thu, 07 Jun 2012 06:43:46 -0400 Subject: [Mailman-Developers] [HyperKitty] Rating of posts In-Reply-To: <1339010663.15279.100.camel@ambre.pingoured.fr> References: <1339010663.15279.100.camel@ambre.pingoured.fr> Message-ID: <4FD085E2.2070604@libertytrek.org> On 2012-06-06 3:24 PM, Pierre-Yves Chibon wrote: > On Wed, 2012-06-06 at 00:43 -0400, Aamir Khan wrote: >> In this case each upvote/downvote will be saved as a separate row in >> database. Your suggestions? > I actually would like to ask you to think at a broader level. > Could you provide us a database schema on how it would look ? > You would have a user table, I assume, the tables for the archives and > I would like you to consider in your design ratings but also the > category and tags assigned to the emails. I would also like to see a way for each user to be able to rate other *users*, not just their individual posts... An example with this list: I would rate Mark Sapiro as '5' (or whatever was designated as 'highest value'), so whenever I sort answers to any particular questions, Marks answers should appear at the top of the results before anyone else's (even if someone else's post had a higher rating than Marks for a particular thread) - then would come the individual posts from other users (who I hadn't yet rated)... Hope that made sense... From jessy at jessykate.com Thu Jun 7 18:31:35 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Thu, 7 Jun 2012 09:31:35 -0700 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: <20120605220944.67b50481@resist.wooz.org> References: <20120605220944.67b50481@resist.wooz.org> Message-ID: thanks mark and barry. i've been traveling so apologies for the delayed reply... using `bin/mailman shell` command does indeed work flawlessly, thank you! should the docs be updated perhaps? something either under "getting started with GNU Maiman" or even a specific page on "interacting with mailman through the python interpreter"? i am happy to add something or contribute to that; i think it would certainly facilitate others working with the software if that bit was a bit more clear. in general, what is that the recommended way to script one's own interactions with mailman as part of a larger python program? would it be through mailman.client "official bindings"? i see there are a number of imports in bin/mailman and subsequently in src/mailman/commands/cli_withlist.py (which i gather is an alias for "shell"). should those be a sufficient set of imports and initializations? in this particular case, i am building a django app so it seems i can rely on postorious and mailman.client, but that wouldn't necessarily be the case in general, and my curiosity is piqued about how the scaffolding process and imports work so i can do things The Right Way. thanks! jessy On Tue, Jun 5, 2012 at 7:09 PM, Barry Warsaw wrote: > Hi Jessy, > > You're doing everything right, but there's one conceptual step that you're > missing, and which isn't evident from the docs. > > As Mark says, `mailman shell` is the best way to get a Python interactive > interpreter shell to play with the internal Mailman API. Why is this > better > than just running the virtualenv's `python` interpreter directly? It's > because there are a bunch of subsystems that have to be initialized before > things will work correctly. E.g., the Zope component architecture (ZCA), > the > logging subsystem, the configuration subsystem, queues, rules, pipelines, > etc. The exception you're getting is because the ZCA hasn't been > initialized > yet. > > `mailman shell` ensures that everything is initialized, then it gives you > an > interpreter prompt. > > Hope that helps. > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/jessy%40jessykate.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Jessy http://jessykate.com From barry at list.org Thu Jun 7 19:16:53 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 7 Jun 2012 13:16:53 -0400 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: References: <20120605220944.67b50481@resist.wooz.org> Message-ID: <20120607131653.08be0b20@resist.wooz.org> On Jun 07, 2012, at 09:31 AM, Jessy Kate Schingler wrote: >should the docs be updated perhaps? something either under "getting started >with GNU Maiman" or even a specific page on "interacting with mailman >through the python interpreter"? i am happy to add something or contribute >to that; i think it would certainly facilitate others working with the >software if that bit was a bit more clear. Hi Jessy. Yes, I definitely think the docs should be updated, but perhaps not the page you were reading. What I've always thought is that overview, architectural, and other useful higher level documentation should go in src/mailman/docs. Still using the reST format, but they needn't necessarily be doctests. So I think an overview page about "interacting with Mailman through the Python interpreter" would make a great page for this directory, and if you want to do a branch/merge proposal on that, I would gladly review it. >in general, what is that the recommended way to script one's own >interactions with mailman as part of a larger python program? would it be >through mailman.client "official bindings"? i see there are a number of >imports in bin/mailman and subsequently in >src/mailman/commands/cli_withlist.py (which i gather is an alias for >"shell"). should those be a sufficient set of imports and initializations? Well, kind of the point is that there are multiple ways of doing it, depending on what you're trying to do. One of the principles I've tried to hold to (successfully or not ;) is that the edges of the system, e.g. the shell scripts and the REST API, should have very little logic outside of managing that edge. So for example, the cli_*.py modules should do little more than parse command line options, massage data into the right format, and then call into the core API, by using the zope.interfaces. By organizing things this way, a Python program could conceivably add the proper directory to its sys.path and just 'import mailman' and various submodules to do exactly the same operations. This wasn't the case with Mailman 2, because the CGIs, cli scripts, etc. had way too much logic in them, so to reproduce their functionality on other boundaries was just too difficult. I think that's a big reason why people wrote shell scripts to use the MM2 cli rather than bin/withlist scripts or importing the Mailman namespace. So, I would say that if you're interacting with MM3 across process boundaries, e.g. Django on one machine with the core on another, the REST API is the best way to go. If you're interacting with MM3 on the same machine, within process, then 'import mailman' could be a good way to go (but of course the REST API still works), or you could use withlist scripts, or just script the cli. (Note that mailman.client, while official, isn't strictly necessary, since the REST API can be used from anything that speaks HTTP.) >in this particular case, i am building a django app so it seems i can rely >on postorious and mailman.client, but that wouldn't necessarily be the case >in general, and my curiosity is piqued about how the scaffolding process >and imports work so i can do things The Right Way. I hope the above explains things, if not just ask! Note that if you go the 'import mailman' route, you will need to initialize the system. A good model for how that's done is src/mailman/bin/mailman.py which is what implements the 'bin/mailman' uber-command. Other than doing fancy things to get --help looking decent, it's almost ridiculously simple. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From richard at NFSNet.org Thu Jun 7 19:26:33 2012 From: richard at NFSNet.org (Richard Wackerbarth) Date: Thu, 7 Jun 2012 12:26:33 -0500 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: References: <20120605220944.67b50481@resist.wooz.org> Message-ID: Jessy, I'm sure that any additions to the documentation will be welcomed. Good end-user documentation always seems to be something that gets shorted by the developers intimately related to the implementation aspect of the code. And, even when they make the effort, I'm not sure that they are the most qualified to write that kind of documentation. Someone less involved in the implementation often has a better perspective of the end-user needs regarding documentation. As for your project, we have assumed that the website serving the UI is not the same host as that which hosts the MM core itself. As a result, attempting to use the command line utilities as your access method would not be the best implementation choice. There is a REST interface exposed, and your access should go through that. Since you are working in the Django world, we would hope that you would utilize the postorius app, and leverage off of it for your customization. By doing so, it will be much easier for you to remain integrated with any changes that come down the line. IMHO, at the moment, the postorius templates can be improved by doing some refactoring to keep their functionality in a {% block %} structure that can be integrated into another site without having to re-implement entire pages. I would like to have you review the present template structure and suggest which portions you would utilize as presented and those which ones you would prefer to change. Thanks for your interest, Richard On Jun 7, 2012, at 11:31 AM, Jessy Kate Schingler wrote: > thanks mark and barry. i've been traveling so apologies for the delayed > reply... > > using `bin/mailman shell` command does indeed work flawlessly, thank you! > > should the docs be updated perhaps? something either under "getting started > with GNU Maiman" or even a specific page on "interacting with mailman > through the python interpreter"? i am happy to add something or contribute > to that; i think it would certainly facilitate others working with the > software if that bit was a bit more clear. > > in general, what is that the recommended way to script one's own > interactions with mailman as part of a larger python program? would it be > through mailman.client "official bindings"? i see there are a number of > imports in bin/mailman and subsequently in > src/mailman/commands/cli_withlist.py (which i gather is an alias for > "shell"). should those be a sufficient set of imports and initializations? > > in this particular case, i am building a Django app so it seems i can rely > on postorious and mailman.client, but that wouldn't necessarily be the case > in general, and my curiosity is piqued about how the scaffolding process > and imports work so i can do things The Right Way. > > thanks! > jessy > > On Tue, Jun 5, 2012 at 7:09 PM, Barry Warsaw wrote: > >> Hi Jessy, >> >> You're doing everything right, but there's one conceptual step that you're >> missing, and which isn't evident from the docs. >> >> As Mark says, `mailman shell` is the best way to get a Python interactive >> interpreter shell to play with the internal Mailman API. Why is this >> better >> than just running the virtualenv's `python` interpreter directly? It's >> because there are a bunch of subsystems that have to be initialized before >> things will work correctly. E.g., the Zope component architecture (ZCA), >> the >> logging subsystem, the configuration subsystem, queues, rules, pipelines, >> etc. The exception you're getting is because the ZCA hasn't been >> initialized >> yet. >> >> `mailman shell` ensures that everything is initialized, then it gives you >> an >> interpreter prompt. >> >> Hope that helps. >> -Barry >> _______________________________________________ >> Mailman-Developers mailing list >> Mailman-Developers at python.org >> http://mail.python.org/mailman/listinfo/mailman-developers >> Mailman FAQ: http://wiki.list.org/x/AgA3 >> Searchable Archives: >> http://www.mail-archive.com/mailman-developers%40python.org/ >> Unsubscribe: >> http://mail.python.org/mailman/options/mailman-developers/jessy%40jessykate.com >> >> Security Policy: http://wiki.list.org/x/QIA9 >> > > > > -- > Jessy > http://jessykate.com > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/richard%40nfsnet.org > > Security Policy: http://wiki.list.org/x/QIA9 From syst3m.w0rm at gmail.com Thu Jun 7 19:42:59 2012 From: syst3m.w0rm at gmail.com (Aamir Khan) Date: Thu, 7 Jun 2012 13:42:59 -0400 Subject: [Mailman-Developers] [HyperKitty] Rating of posts In-Reply-To: <4FD0E4D5.6000308@libertytrek.org> References: <1339010663.15279.100.camel@ambre.pingoured.fr> <4FD085E2.2070604@libertytrek.org> <4FD0E4D5.6000308@libertytrek.org> Message-ID: ok. I think we both are talking about the same thing. And, I totally agree with your point that there should also be a user level rating as well, but the question is how do we rate users, one way is to calculate using number of upvotes and downvotes on his/her answers. Any other suggestion? I think ultimately HyperKitty should have both user level and email level rating but I think email level rating is more basic and I think it's better to implement it first. On Thu, Jun 7, 2012 at 1:28 PM, Tanstaafl wrote: > Hi Aamir, > > I think you misunderstood what I was saying... see below... > > On 2012-06-07 1:14 PM, Aamir Khan wrote: > > On Thu, Jun 7, 2012 at 6:43 AM, Tanstaafl wrote: >> >>> I would also like to see a way for each user to be able to rate >>> other *users*, not just their individual posts... >>> >>> An example with this list: I would rate Mark Sapiro as '5' (or >>> whatever was designated as 'highest value'), so whenever I sort >>> answers to any particular questions, Marks answers should appear at >>> the top of the results before anyone else's (even if someone else's >>> post had a higher rating than Marks for a particular thread) - then >>> would come the individual posts from other users (who I hadn't yet >>> rated)... >>> >>> Hope that made sense... >>> >> > I don't completely agree with your point. Let's say Mark posts good >> emails to the list then he will obviously get a higher rating by users. >> > > My understanding was the initial plan was only to allow for rating > individual *posts*, not the *posters* - so, in your example, Mark *himself* > will not have a rating (unless you provided a way to basically summarize an > average of all of the ratings of all of each users posts)... > > What I was talking about, is a way for me to rate Mark for my own > *personal* benefit... > > > The primary purpose of archiver is to serve people who are not >> subscribed to the mailing list. >> > > I disagree... > > The primary purpose of the archiver is to archive the list messages. Some > lists *and list archives) are public, some are not. > > > They don't even know who Mark is >> > > They do if they are a member of the list for any amount of time. > > > and if they encounter a particular question lets say "How to setup >> Mailman3 in virtualenv" then they would like to see the best answer >> and not the answer of person who generally posts good answer to the >> list. >> > > Again, I think you misunderstood... and again - I was talking about a way > for me to rate Mark, which would be stored as one of *my* preferences... > so, only *I* would benefit from rating him highly - no one else would know > that I had rated him highly. > > Think of it as a way for me to filter answers to questions based on who > *I* trust and am interested in hearing from. > > I'd also like a way to rate users negatively, so I won't even see their > posts... again, said ratings only having any meaning for me when I'm logged > in. > > Do you agree? >> > > I don't think we are talking about the same thing, so until we both agree > on what we are discussing, I don't think the question is relevant... ;) > -- Aamir Khan | 3rd Year | Computer Science & Engineering | IIT Roorkee From jessy at jessykate.com Mon Jun 11 06:50:08 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Mon, 11 Jun 2012 00:50:08 -0400 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: <20120607131653.08be0b20@resist.wooz.org> References: <20120605220944.67b50481@resist.wooz.org> <20120607131653.08be0b20@resist.wooz.org> Message-ID: hello again, and thanks for the response. What I've always thought is that overview, architectural, and other useful > higher level documentation should go in src/mailman/docs. Still using the > reST format, but they needn't necessarily be doctests. So I think an > overview > page about "interacting with Mailman through the Python interpreter" would > make a great page for this directory, and if you want to do a branch/merge > proposal on that, I would gladly review it. > ok. i'm not totally familiar with the what/where/why of the different doc locations. the docs that get built with sphinx (ironically, the docs about building the docs are actually not quite right :)) are the same as the ones at http://packages.python.org/mailman/, and seem to be a somewhat-overlapping set with those in src/mailman/docs? i am happy to contribute to whichever location you all think is best. > One of the principles I've tried to hold to (successfully or not ;) is that > the edges of the system, e.g. the shell scripts and the REST API, should > have > very little logic outside of managing that edge. So for example, the > cli_*.py > modules should do little more than parse command line options, massage data > into the right format, and then call into the core API, by using the > zope.interfaces. By organizing things this way, a Python program could > conceivably add the proper directory to its sys.path and just 'import > mailman' > and various submodules to do exactly the same operations. > now that's i'm getting a tiny bit more familiar with the system, i can see how this makes sense with so many different interacting components. the zope interfaces seem to be a conceptually distinct part of the system but i can't see where their role is documented. am i missing that? (apologies for asking questions that i'm sure have been answered before, feel free to just point me to a url). > So, I would say that if you're interacting with MM3 across process > boundaries, > e.g. Django on one machine with the core on another, the REST API is the > best > way to go. If you're interacting with MM3 on the same machine, within > process, then 'import mailman' could be a good way to go (but of course the > REST API still works), or you could use withlist scripts, or just script > the > cli. > great, yes, this makes sense. it seems like the REST interface is designed to be the primary interface to the system, which is logical given that MM3 would in general be running as a remote service. i was originally thinking of it more like this: MM3 -> exposes python packages and connectors -> my code |--> rest interface (optional) sort of like hooking into, say, mongodb or a remote mysql instance. whereas IIUC the design is intended to be used more like this: MM3 -> rest interface -> official/unofficial MM3 rest bindings (mailman.client) -> my code (or, alternatively, postorius goes here, etc.) in this interpretation, it's not that hooking right into MM3 is a bad thing per se, but is kind of missing the point, and doing additional work to route around the effort that's been made to provide a simple interface. > I hope the above explains things, if not just ask! Note that if you go the > 'import mailman' route, you will need to initialize the system. A good > model > for how that's done is src/mailman/bin/mailman.py which is what implements > the > 'bin/mailman' uber-command. Other than doing fancy things to get --help > looking decent, it's almost ridiculously simple. > thank you! i think the picture is coming together. i'll keep documenting what i'm doing and hopefully will provide some good install/setup/testing documentation... -- Jessy http://jessykate.com From jessy at jessykate.com Mon Jun 11 06:57:23 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Mon, 11 Jun 2012 00:57:23 -0400 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: References: <20120605220944.67b50481@resist.wooz.org> Message-ID: hi richard, Good end-user documentation always seems to be something that gets shorted > by the developers intimately related to the implementation aspect of the > code. And, even when they make the effort, I'm not sure that they are the > most qualified to write that kind of documentation. Someone less involved > in the implementation often has a better perspective of the end-user needs > regarding documentation. > indeed, fresh perspectives are fleeting :). > Since you are working in the Django world, we would hope that you would > utilize the postorius app, and leverage off of it for your customization. > By doing so, it will be much easier for you to remain integrated with any > changes that come down the line. > agreed and i am looking forward to integrating this. > IMHO, at the moment, the postorius templates can be improved by doing some > refactoring to keep their functionality in a {% block %} structure that can > be integrated into another site without having to re-implement entire > pages. I would like to have you review the present template structure and > suggest which portions you would utilize as presented and those which ones > you would prefer to change. > thanks for the pointers and direction. i'll keep this in mind as i begin working with the code and try to make some useful suggestions (and/or will send links to any code i factor out myself). one question i had about the UI - are there plans for including views/templates for list archives? thanks, -- Jessy http://jessykate.com From richard at nfsnet.org Mon Jun 11 12:21:07 2012 From: richard at nfsnet.org (Richard Wackerbarth) Date: Mon, 11 Jun 2012 05:21:07 -0500 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: References: <20120605220944.67b50481@resist.wooz.org> Message-ID: <31CC81D2-EBEB-48EC-A670-F8E31E09126B@nfsnet.org> Jessy, Yes, there are plans (and activity) related to the presentation of list archives within the django framework. In fact, Aamir, one of our GSoC interns, is working on that very subject as his project for the summer. In keeping with the django philosophy of combining many special purpose "apps" to create the overall website, the archive access will be packaged as a component separate from the postorius administrative interface. Richard On Jun 10, 2012, at 11:57 PM, Jessy Kate Schingler wrote: > one question i had about the UI - are there plans for including views/templates for list archives? From richard at nfsnet.org Mon Jun 11 12:43:33 2012 From: richard at nfsnet.org (Richard Wackerbarth) Date: Mon, 11 Jun 2012 05:43:33 -0500 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs Message-ID: Are we making a design mistake? The current design of the Postorius and Hyperkitty web interfaces to the mailing list and its archives uses the fully qualified list submission email address as a component of the URLs presented to the public. Is this really a good idea? Just think of the exposure that search engines, etc. will give to these email addresses. I fear that doing this will create an even greater invitation to those who harvest email addresses for the purpose of spamming and other nefarious reasons. Additionally, in the most common usage case, it makes the URL significantly longer than it needs to be. In most cases, the website address determines the email domain of the associated lists. Only a few websites are serving mailing lists from multiple email domains. Those sites would need to have some mechanism to unambiguiously identify the list being referenced. But for most sites, the common name of the list is sufficient. One of the design principles of Django is that the website designer can present his content by way of URLs of his choosing. Presenting the actual email address of a list may "leak" information that the user wishes to obscure. I think that we should rethink this decision and follow a "slug" approach to the identification of the mailing lists in URLs. Those who choose to do so can use the fqdn as their slug. But others would be able to readily change the mapping without having to rewrite significant parts of the interface code. Comments? From jessy at jessykate.com Mon Jun 11 15:28:40 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Mon, 11 Jun 2012 09:28:40 -0400 Subject: [Mailman-Developers] Mailman 3.0 fresh install - problems from python interpreter In-Reply-To: <31CC81D2-EBEB-48EC-A670-F8E31E09126B@nfsnet.org> References: <20120605220944.67b50481@resist.wooz.org> <31CC81D2-EBEB-48EC-A670-F8E31E09126B@nfsnet.org> Message-ID: good to know, look forward to seeing it! On Mon, Jun 11, 2012 at 6:21 AM, Richard Wackerbarth wrote: > Jessy, > > Yes, there are plans (and activity) related to the presentation of list > archives within the django framework. > In fact, Aamir, one of our GSoC interns, is working on that very subject > as his project for the summer. > > In keeping with the django philosophy of combining many special purpose > "apps" to create the overall website, the archive access will be packaged > as a component separate from the postorius administrative interface. > > Richard > > On Jun 10, 2012, at 11:57 PM, Jessy Kate Schingler wrote: > > one question i had about the UI - are there plans for including > views/templates for list archives? > > -- Jessy http://jessykate.com From a.badger at gmail.com Mon Jun 11 16:51:05 2012 From: a.badger at gmail.com (Toshio Kuratomi) Date: Mon, 11 Jun 2012 07:51:05 -0700 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: References: Message-ID: <20120611145105.GT2569@unaka.lan> On Mon, Jun 11, 2012 at 05:43:33AM -0500, Richard Wackerbarth wrote: > Are we making a design mistake? > > The current design of the Postorius and Hyperkitty web interfaces to the mailing list and its archives uses the fully qualified list submission email address as a component of the URLs presented to the public. > > Is this really a good idea? Just think of the exposure that search engines, etc. will give to these email addresses. I fear that doing this will create an even greater invitation to those who harvest email addresses for the purpose of spamming and other nefarious reasons. > > Additionally, in the most common usage case, it makes the URL significantly longer than it needs to be. In most cases, the website address determines the email domain of the associated lists. Only a few websites are serving mailing lists from multiple email domains. Those sites would need to have some mechanism to unambiguiously identify the list being referenced. But for most sites, the common name of the list is sufficient. > > One of the design principles of Django is that the website designer can present his content by way of URLs of his choosing. > > Presenting the actual email address of a list may "leak" information that the user wishes to obscure. > > I think that we should rethink this decision and follow a "slug" approach to the identification of the mailing lists in URLs. Those who choose to do so can use the fqdn as their slug. But others would be able to readily change the mapping without having to rewrite significant parts of the interface code. > > Comments? > I don't think I buy into the obscuring of information argument because the mailing list already requires you to know the fqdn to send email to the list but I definitely do see the convenience factor of having shorter slugs for sites without lists for multiple domains. A slug would be possible but probably should be defined at the mailman3 core level similar to how the stable URL hash for emails is defined there. Otherwise the list administrator has to enter it in multiple places and it can be different between one app and another. If I recall correctly, I was asked by mailman3 for an unadorned version of the list name as well as the fqdn when I set up a list. So that could be used if the administrator knows that there's no danger of collisions. But how does the administrator know that? I think that it's probably the person who sets up postorius and the archiver rather than the person that sets up mailman3 core that knows this (after all, in a distant future, we could have webui's and archivers that can aggregate multiple mailman3 servers transparently for large sites with multiple departments). So perhaps we should have the front ends, not core, attempt to resolve non-fqdn listaddresses. If I'm given mailman-developers in my url, I do a search for ^mailman-developers at .* and if it comes up with one entry I redirect to the fqdn (since I don't think that obscuring is necessary here, a redirect seems appropriate). If I come up with multiple entries, I ask the user to choose from the list of possibilities. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From stephen at xemacs.org Mon Jun 11 17:48:28 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 12 Jun 2012 00:48:28 +0900 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: References: Message-ID: <874nqh97eb.fsf@uwakimon.sk.tsukuba.ac.jp> Richard Wackerbarth writes: > The current design of the Postorius and Hyperkitty web interfaces > to the mailing list and its archives uses the fully qualified list > submission email address as a component of the URLs presented to > the public. > > I fear that doing this will create an even greater invitation to > those who harvest email addresses for the purpose of spamming and > other nefarious reasons. I think this is overblown. These email addresses are almost certainly easily available to spammers in other ways if they're going to be visible to the public in the web interfaces. (Consider List-Post, for example.) If we're going to be paranoid about this, we should also refuse to subscribe users with Microsoft browsers and MUAs on the grounds that they're far more likely to have their address books stolen. :-) OTOH, a lot of people do worry about this. We should definitely consider getting those addresses out of the URLs to make it easier for the security-with-obscurity crowd to lock down their sites for any reason they choose. > I think that we should rethink this decision and follow a "slug" > approach to the identification of the mailing lists in URLs. Those > who choose to do so can use the fqdn as their slug. But others > would be able to readily change the mapping without having to > rewrite significant parts of the interface code. +1 It might be worth reviewing all the uses of URLs to see which ones can be dispensed with (ie, have such "slugs" substituted), and which ones are essential to functioning of Mailman (eg, List-Post, which may be suppressed at list-owner option, but if not, must contain the posting address). From barry at list.org Mon Jun 11 18:28:41 2012 From: barry at list.org (Barry Warsaw) Date: Mon, 11 Jun 2012 12:28:41 -0400 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: References: Message-ID: <20120611122841.2f4088c0@resist.wooz.org> On Jun 11, 2012, at 05:43 AM, Richard Wackerbarth wrote: >I think that we should rethink this decision and follow a "slug" approach to >the identification of the mailing lists in URLs. Those who choose to do so >can use the fqdn as their slug. But others would be able to readily change >the mapping without having to rewrite significant parts of the interface >code. I agree that the shorter-url argument is more compelling than the security-by-obscurity argument. If the mapping need only happen in one direction, then something as simple as hashing the mlist fqdn and taking a substring would probably be good enough. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From richard at nfsnet.org Mon Jun 11 23:58:32 2012 From: richard at nfsnet.org (Richard Wackerbarth) Date: Mon, 11 Jun 2012 16:58:32 -0500 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <20120611122841.2f4088c0@resist.wooz.org> References: <20120611122841.2f4088c0@resist.wooz.org> Message-ID: Actually, the Django methodology would allow the website designer to choose any URL that she wishes. So, Postorius should be able to set a configuration parameter to any value that the designer chooses and "core" would just be echoing it. Richard On Jun 11, 2012, at 11:28 AM, Barry Warsaw wrote: > On Jun 11, 2012, at 05:43 AM, Richard Wackerbarth wrote: > >> I think that we should rethink this decision and follow a "slug" approach to >> the identification of the mailing lists in URLs. Those who choose to do so >> can use the fqdn as their slug. But others would be able to readily change >> the mapping without having to rewrite significant parts of the interface >> code. > > I agree that the shorter-url argument is more compelling than the > security-by-obscurity argument. > > If the mapping need only happen in one direction, then something as simple as > hashing the mlist fqdn and taking a substring would probably be good enough. > > -Barry From stephen at xemacs.org Tue Jun 12 05:06:44 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 12 Jun 2012 12:06:44 +0900 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <20120611122841.2f4088c0@resist.wooz.org> References: <20120611122841.2f4088c0@resist.wooz.org> Message-ID: <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> Barry Warsaw writes: > hashing the mlist fqdn and taking a substring would probably be good enough. Ewwwwwwwwww! -1 From f at state-of-mind.de Tue Jun 12 12:16:38 2012 From: f at state-of-mind.de (Florian Fuchs) Date: Tue, 12 Jun 2012 12:16:38 +0200 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> References: <20120611122841.2f4088c0@resist.wooz.org> <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4FD71706.4080100@state-of-mind.de> Hi, 1) We could change this part of the URLs to something like: . This way we can compose the full fqdn_listname from the URL but it's not as obvious as the list's email address (it's also still readable). 2) Adding to Richard's slug idea: We could provide a "slug"-field in the list settings (and create list) form. The default slug is the format described in 1) - but an admin can change it to any custom (and shorter) slug they like (as long as it's unique). I suggest that we save the slug/fqdn_listname mapping in a local db table (it's only a ui specific setting, so the core doesn't need to know about it). Cheers Florian Am 12.06.12 05:06, schrieb Stephen J. Turnbull: > Barry Warsaw writes: > > > hashing the mlist fqdn and taking a substring would probably be good enough. > > Ewwwwwwwwww! > > -1 > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/f%40state-of-mind.de > > Security Policy: http://wiki.list.org/x/QIA9 From richard at NFSNet.org Tue Jun 12 15:01:08 2012 From: richard at NFSNet.org (Richard Wackerbarth) Date: Tue, 12 Jun 2012 08:01:08 -0500 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <4FD71706.4080100@state-of-mind.de> References: <20120611122841.2f4088c0@resist.wooz.org> <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> <4FD71706.4080100@state-of-mind.de> Message-ID: <99AE13B3-7028-471D-9948-F06FC327C0AD@NFSNet.org> On Jun 12, 2012, at 5:16 AM, Florian Fuchs wrote: > 1) We could change this part of the URLs to something like: > . This way we can compose the full > fqdn_listname from the URL but it's not as obvious as the list's email > address (it's also still readable). > > 2) Adding to Richard's slug idea: We could provide a "slug"-field in the > list settings (and create list) form. The default slug is the format > described in 1) - but an admin can change it to any custom (and shorter) > slug they like (as long as it's unique). +1 > I suggest that we save the slug/fqdn_listname mapping in a local db > table (it's only a ui specific setting, so the core doesn't need to know > about it). Core will need to be provided a dictionary of items related to the configuration of the list infrastructure. The URL for subscriber account maintenance is one of those items. However, the contents of that field, or any of the others, and how they are derived, is outside the scope of "core". To core, these are just configuration constants. > Cheers > Florian From barry at list.org Tue Jun 12 16:58:21 2012 From: barry at list.org (Barry Warsaw) Date: Tue, 12 Jun 2012 10:58:21 -0400 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <99AE13B3-7028-471D-9948-F06FC327C0AD@NFSNet.org> References: <20120611122841.2f4088c0@resist.wooz.org> <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> <4FD71706.4080100@state-of-mind.de> <99AE13B3-7028-471D-9948-F06FC327C0AD@NFSNet.org> Message-ID: <20120612105821.421c6778@resist.wooz.org> On Jun 12, 2012, at 08:01 AM, Richard Wackerbarth wrote: >Core will need to be provided a dictionary of items related to the >configuration of the list infrastructure. The URL for subscriber account >maintenance is one of those items. However, the contents of that field, or >any of the others, and how they are derived, is outside the scope of "core". >To core, these are just configuration constants. It does, and it's not difficult to add new variables, except for the nightmare (currently ongoing) of migrating the SQLite schema. -Barry From richard at NFSNet.org Tue Jun 12 17:19:07 2012 From: richard at NFSNet.org (Richard Wackerbarth) Date: Tue, 12 Jun 2012 10:19:07 -0500 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <20120612105821.421c6778@resist.wooz.org> References: <20120611122841.2f4088c0@resist.wooz.org> <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> <4FD71706.4080100@state-of-mind.de> <99AE13B3-7028-471D-9948-F06FC327C0AD@NFSNet.org> <20120612105821.421c6778@resist.wooz.org> Message-ID: <3C60827C-BE9E-490B-8BE4-77B30E3C8FFE@NFSNet.org> Perhaps you should consider a different strategy that does not couple so tightly the SQL schema with the parameters. Rather that having an object which tries to keep everything directly on the object, place many of those values in a dictionary and do (key, value) lookups on the dictionary rather expecting the key to be a property of the object itself. That way, additional values can be associated with the object without changing the underlying database schema. Richard On Jun 12, 2012, at 9:58 AM, Barry Warsaw wrote: > On Jun 12, 2012, at 08:01 AM, Richard Wackerbarth wrote: > >> Core will need to be provided a dictionary of items related to the >> configuration of the list infrastructure. The URL for subscriber account >> maintenance is one of those items. However, the contents of that field, or >> any of the others, and how they are derived, is outside the scope of "core". >> To core, these are just configuration constants. > > It does, and it's not difficult to add new variables, except for the nightmare > (currently ongoing) of migrating the SQLite schema. From barry at list.org Tue Jun 12 17:24:06 2012 From: barry at list.org (Barry Warsaw) Date: Tue, 12 Jun 2012 11:24:06 -0400 Subject: [Mailman-Developers] MM3 - Using fqdn in the exposed URLs In-Reply-To: <3C60827C-BE9E-490B-8BE4-77B30E3C8FFE@NFSNet.org> References: <20120611122841.2f4088c0@resist.wooz.org> <87pq956xff.fsf@uwakimon.sk.tsukuba.ac.jp> <4FD71706.4080100@state-of-mind.de> <99AE13B3-7028-471D-9948-F06FC327C0AD@NFSNet.org> <20120612105821.421c6778@resist.wooz.org> <3C60827C-BE9E-490B-8BE4-77B30E3C8FFE@NFSNet.org> Message-ID: <20120612112406.2ef1e855@resist.wooz.org> On Jun 12, 2012, at 10:19 AM, Richard Wackerbarth wrote: >Perhaps you should consider a different strategy that does not couple so >tightly the SQL schema with the parameters. > >Rather that having an object which tries to keep everything directly on the >object, place many of those values in a dictionary and do (key, value) >lookups on the dictionary rather expecting the key to be a property of the >object itself. > >That way, additional values can be associated with the object without >changing the underlying database schema. It's a good idea, and I've thought about that. The Pendings database is essentially that. Of course, switching to it is itself a schema migration. ;) Maybe I should just renege on my beta1 promise ;). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From mark at msapiro.net Fri Jun 15 19:04:59 2012 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 15 Jun 2012 10:04:59 -0700 Subject: [Mailman-Developers] Mailman 2.1.15 final released. Message-ID: <4FDB6B3B.7070506@msapiro.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am happy to announce the final release of Mailman 2.1.15. This release is identical to the 2.1.15rc1 release except for the version number and the inclusion of a missing part of the HTML installation manual. Python 2.4 is the minimum supported, but Python 2.6 is recommended. This release should work with Python 2.7, but has not been tested with that version. This release includes minor security enhancements, new features and bug fixes. See the Changelog at for more details. Mailman is free software for managing email mailing lists and e-newsletters. Mailman is used for all the python.org and SourceForge.net mailing lists, as well as at hundreds of other sites. For more information, please see: http://www.list.org http://www.gnu.org/software/mailman http://mailman.sourceforge.net/ Mailman 2.1.15 can be downloaded from https://launchpad.net/mailman/2.1/ http://ftp.gnu.org/gnu/mailman/ https://sourceforge.net/projects/mailman/ - -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFP22s7VVuXXpU7hpMRAmqOAJ9rDBhSkYDW2yiaqljJzjQtPX8xgwCfXpS7 4I6YOtQjk4tn/UGrtyCGgIQ= =rsB+ -----END PGP SIGNATURE----- From tanstaafl at libertytrek.org Fri Jun 15 21:38:39 2012 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Fri, 15 Jun 2012 15:38:39 -0400 Subject: [Mailman-Developers] Mailman 2.1.15 final released. In-Reply-To: <4FDB6B3B.7070506@msapiro.net> References: <4FDB6B3B.7070506@msapiro.net> Message-ID: <4FDB8F3F.2050001@libertytrek.org> On 2012-06-15 1:04 PM, Mark Sapiro wrote: > Python 2.4 is the minimum supported, but Python 2.6 is recommended. > This release should work with Python 2.7, but has not been tested with > that version. I'm curious about why 2.7 isn't officially supported (current stable is already at 3.2.x)? I've been running 2.1.14 with 2.7 (now at 2.7.3) since it went stable (I'm on gentoo) with zero problems... but am I asking for trouble? From mark at msapiro.net Fri Jun 15 22:28:39 2012 From: mark at msapiro.net (Mark Sapiro) Date: Fri, 15 Jun 2012 13:28:39 -0700 Subject: [Mailman-Developers] Mailman 2.1.15 final released. In-Reply-To: <4FDB8F3F.2050001@libertytrek.org> Message-ID: Tanstaafl wrote: >On 2012-06-15 1:04 PM, Mark Sapiro wrote: >> Python 2.4 is the minimum supported, but Python 2.6 is recommended. >> This release should work with Python 2.7, but has not been tested with >> that version. > >I'm curious about why 2.7 isn't officially supported (current stable is >already at 3.2.x)? I have not gotten around to installing Python 2.7 on the machines that I use for Mailman 2.x development, testing and production. That's why I have to say that this release "should work with Python 2.7, but has not been tested with that version." I can't recommend running Mailman with a Python version that I haven't run it on. >I've been running 2.1.14 with 2.7 (now at 2.7.3) since it went stable >(I'm on gentoo) with zero problems... but am I asking for trouble? I don't think so. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From andrew at hodgsonfamily.org Fri Jun 15 21:46:43 2012 From: andrew at hodgsonfamily.org (Andrew Hodgson) Date: Fri, 15 Jun 2012 19:46:43 +0000 Subject: [Mailman-Developers] Mailman 2.1.15 final released. In-Reply-To: <4FDB6B3B.7070506@msapiro.net> References: <4FDB6B3B.7070506@msapiro.net> Message-ID: Mark Sapiro wrote: >I am happy to announce the final release of Mailman 2.1.15. This release is identical to the 2.1.15rc1 release except for the version number and >the inclusion of a missing part of the HTML installation manual. Thanks for this as ever, quality release. I upgraded in a very short space of time, all working well. Andrew. From tanstaafl at libertytrek.org Fri Jun 15 22:51:29 2012 From: tanstaafl at libertytrek.org (Tanstaafl) Date: Fri, 15 Jun 2012 16:51:29 -0400 Subject: [Mailman-Developers] Mailman 2.1.15 final released. In-Reply-To: References: Message-ID: <4FDBA051.9090407@libertytrek.org> On 2012-06-15 4:28 PM, Mark Sapiro wrote: > Tanstaafl wrote: >> I've been running 2.1.14 with 2.7 (now at 2.7.3) since it went stable >> (I'm on gentoo) with zero problems... but am I asking for trouble? > I don't think so. Ok, thanks - and as always, thanks very much for mailman! I'm really looking forward to the 3.0 release... From fmouse-mailman at fmp.com Mon Jun 18 05:28:24 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Sun, 17 Jun 2012 22:28:24 -0500 Subject: [Mailman-Developers] Personal patch Message-ID: <1339990104.26588.23.camel@pudina.fmp.com> Can someone give me some feedback on the following patch to SMTPDirect.py - whatever I've overlooked, or done that might be dangerous? The purpose of this patch is to insert a header, "X-subdata" into VERPed emails which won't be flagged and redacted by AOL's brain-dead "Email Feedback Report" system, and will continue to allow my local scripts to unsubscribe subscribers who hit the "Report Spam" button on their AOL mail UI. The content of the header is an MD5 hash of the receiving subscriber's email address - the same information contained in the Sender and Return-path headers, normally munged ("redacted") by AOL. The hope is that this hash will address AOL's privacy concerns, and/or else fall beneath the intelligence level of their scrutiny. The address hash can be compared against the list of subscribers to the list, identified in several (improperly redacted or un-redacted) headers. I'm not submitting this as a suggested patch for Mailman, but just asking for some feedback from people who know the code better than I do. --- SMTPDirect.py.orig 2012-06-17 17:16:25.000000000 -0500 +++ SMTPDirect.py 2012-06-17 21:17:25.000000000 -0500 @@ -43,6 +43,8 @@ from email.Utils import formataddr from email.Header import Header from email.Charset import Charset +from md5crypt import md5crypt +from random import choice DOT = '.' @@ -307,6 +309,9 @@ 'host' : DOT.join(rdomain), } envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) + saltmarsh = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrsyuvwxyz1234567890./" + if not msgdata.has_key("X-subdata"): + msgcopy["X-Subdata"] = md5crypt(rmailbox + "@" + DOT.join(rdomain), choice(saltmarsh) + choice(saltmarsh)) if mlist.personalize == 2: # When fully personalizing, we want the To address to point to the # recipient, not to the mailing list From mark at msapiro.net Mon Jun 18 06:22:41 2012 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 17 Jun 2012 21:22:41 -0700 Subject: [Mailman-Developers] Personal patch In-Reply-To: <1339990104.26588.23.camel@pudina.fmp.com> Message-ID: Lindsay Haisley wrote: >Can someone give me some feedback on the following patch to >SMTPDirect.py - whatever I've overlooked, or done that might be >dangerous? [...] >--- SMTPDirect.py.orig 2012-06-17 17:16:25.000000000 -0500 >+++ SMTPDirect.py 2012-06-17 21:17:25.000000000 -0500 >@@ -43,6 +43,8 @@ > from email.Utils import formataddr > from email.Header import Header > from email.Charset import Charset >+from md5crypt import md5crypt >+from random import choice > > DOT = '.' > >@@ -307,6 +309,9 @@ > 'host' : DOT.join(rdomain), > } > envsender = '%s@%s' % ((mm_cfg.VERP_FORMAT % d), DOT.join(bdomain)) >+ saltmarsh = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrsyuvwxyz1234567890./" >+ if not msgdata.has_key("X-subdata"): >+ msgcopy["X-Subdata"] = md5crypt(rmailbox + "@" + DOT.join(rdomain), choice(saltmarsh) + choice(saltmarsh)) rmailbox + "@" + DOT.join(rdomain) just does the inverse of rmailbox, rdomain = Utils.ParseEmail(recip) So why not just make the above >+ msgcopy["X-Subdata"] = md5crypt(recip, choice(saltmarsh) + choice(saltmarsh)) Other than that, it looks OK assuming there is an appropriate md5crypt module in Mailman's path. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From fmouse-mailman at fmp.com Mon Jun 18 17:20:41 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Mon, 18 Jun 2012 10:20:41 -0500 Subject: [Mailman-Developers] Personal patch In-Reply-To: References: Message-ID: <1340032841.26588.52.camel@pudina.fmp.com> On Sun, 2012-06-17 at 21:22 -0700, Mark Sapiro wrote: > rmailbox + "@" + DOT.join(rdomain) > > just does the inverse of > > rmailbox, rdomain = Utils.ParseEmail(recip) > > So why not just make the above > > >+ msgcopy["X-Subdata"] = md5crypt(recip, choice(saltmarsh) + choice(saltmarsh)) > Thanks, Mark. I discovered that rdomain is a list and couldn't use it as-is, and I assumed that recip was a data structure other than a simple string and contained the subscriber name, but apparently it's only used as an index to look up the name. > Other than that, it looks OK assuming there is an appropriate md5crypt > module in Mailman's path. md5crypt is a Python module distributed, in Ubuntu and Mint at least, with the Landscape admin tool. It seems that crypt.crypt is limited to 8 characters in cleartext and is therefore mostly useful for passwords. Any function that will generate a hash of an arbitrarily long email address would do. A simpler, better choice might be hashlib.md5(recip).hexdigest(), and no need to mess with a salt, or a special purpose module. Strong security isn't an issue. I only need a one-way hash that's quick to generate and easily reproducible from cleartext. -- Lindsay Haisley | "The only unchanging certainty FMP Computer Services | is the certainty of change" 512-259-1190 | http://www.fmp.com | - Ancient wisdom, all cultures From fmouse-mailman at fmp.com Thu Jun 21 00:37:42 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Wed, 20 Jun 2012 17:37:42 -0500 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 Message-ID: <1340231862.24398.10.camel@pudina.fmp.com> I'm trying to install mm 3.0.0b1 as per the directions in START.rst. I'm running into a bit of a problem with zc.buildout. in the bootstrap.py script. The script reports: $ python bootstrap.py Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg Traceback (most recent call last): File "bootstrap.py", line 254, in ws.require(requirement) File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 666, in require File "/tmp/tmp6ebmh9/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 565, in resolve pkg_resources.DistributionNotFound: zc.buildout==1.5.2 However; $ dpkg -l python-zc.buildout Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Description +++-==================================-==================================-== ii python-zc.buildout 1.5.2-1 system for managing development buildouts I explicitly installed it! Do I need to update the mm3 package? Where do I go from here? -- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com | From stephen at xemacs.org Thu Jun 21 08:57:21 2012 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 21 Jun 2012 15:57:21 +0900 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340231862.24398.10.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> Message-ID: <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> Lindsay Haisley writes: > pkg_resources.DistributionNotFound: zc.buildout==1.5.2 Butbutbutbutbut: > ii python-zc.buildout 1.5.2-1 system for managing development buildouts Hm. I suspect that Debian has installed zc.buildout for a different version of Python. Zope is extremely finicky about that. From jessy at jessykate.com Thu Jun 21 09:00:20 2012 From: jessy at jessykate.com (Jessy Kate Schingler) Date: Thu, 21 Jun 2012 09:00:20 +0200 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1340231862.24398.10.camel@pudina.fmp.com> <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: FWIW, i found installing within a virtualenv to be a sanity-inducing operation for very similar reasons. On Thu, Jun 21, 2012 at 8:57 AM, Stephen J. Turnbull wrote: > Lindsay Haisley writes: > > > pkg_resources.DistributionNotFound: zc.buildout==1.5.2 > > Butbutbutbutbut: > > > ii python-zc.buildout 1.5.2-1 > system for managing development buildouts > > Hm. I suspect that Debian has installed zc.buildout for a different > version of Python. Zope is extremely finicky about that. > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: > http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: > http://mail.python.org/mailman/options/mailman-developers/jessy%40jessykate.com > > Security Policy: http://wiki.list.org/x/QIA9 > -- Jessy http://jessykate.com From fmouse-mailman at fmp.com Thu Jun 21 16:25:27 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Thu, 21 Jun 2012 09:25:27 -0500 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> References: <1340231862.24398.10.camel@pudina.fmp.com> <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1340288727.16746.24.camel@pudina.fmp.com> On Thu, 2012-06-21 at 15:57 +0900, Stephen J. Turnbull wrote: > Hm. I suspect that Debian has installed zc.buildout for a different > version of Python. Zope is extremely finicky about that. > well...., $ python2.6 Python 2.6.7 (r267:88850, Aug 11 2011, 12:18:09) <---- Version 2.6 [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from zc.buildout import * >>> .... :) and....., $ python2.7 Python 2.7.2+ (default, Oct 4 2011, 20:06:09) <----- Version 2.7 [GCC 4.6.1] on linux2 >>> from zc.buildout import * >>> .... :) zc.buildout is installed for both py 2.6 and 2.7. Could zope be barfing on the full package version spec, "2.7.2-7ubuntu2"? Jessy Kate Schingler said: > FWIW, i found installing within a virtualenv to be a sanity-inducing > operation for very similar reasons. > What's involved here? The target box for the install is my desktop workstation, Linux Mint (a Ubuntu knock-off) running on a virtual machine; not otherwise an issue. I don't grok "virtualenv", nor zope either, having never really encountered either one. I _do_ know enough about python to offer some possibly productive suggestion and patches to the mm code itself, but I don't know if I have the personal bandwidth to learn a couple of new technologies and debug the install system in order to do this. Jessy Kate, can you send me an online reference on setting up a proper virtual environment? -- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com | From barry at list.org Thu Jun 21 16:36:09 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 21 Jun 2012 10:36:09 -0400 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340231862.24398.10.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> Message-ID: <20120621103609.0a9fb13d@limelight.wooz.org> On Jun 20, 2012, at 05:37 PM, Lindsay Haisley wrote: >I'm trying to install mm 3.0.0b1 as per the directions in START.rst. START.rst needs to be updated. In general, buildout is only needed to run the test suite, so I think it's more appropriate for development rather than deployment. In fact, I'd like to get rid of the need for buildout and just go with virtualenv, but the test suite currently depends on the buildout environment, so that's not yet feasible. For deployments (i.e. you just want to run mm3) it's better to create a virtualenv and then `python setup.py install` into that virtualenv. I should note that on my Ubuntu development machines, I do not have python-zc.buildout installed, but instead after running `python bootstrap.py` I end up with the zc.buildout-1.5.2-pyX.Y.egg in my eggs directory. So I'm guessing even though the versions are the same, there's some other incompatibility between the Debuntu package and the PyPI package, but I don't know what that might be. Cheers, -Barry From fmouse-mailman at fmp.com Thu Jun 21 17:32:25 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Thu, 21 Jun 2012 10:32:25 -0500 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <20120621103609.0a9fb13d@limelight.wooz.org> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> Message-ID: <1340292745.16746.32.camel@pudina.fmp.com> On Thu, 2012-06-21 at 10:36 -0400, Barry Warsaw wrote: > For deployments (i.e. you just want to run mm3) it's better to create a > virtualenv and then `python setup.py install` into that virtualenv. Barry, I don't know if you followed the discussion on the Users list regarding placing an encrypted (AES) header into personalized, VERPed list posts which would contain the subscriber recipient's address and possibly also the list address. This would satisfy AOL's TOS (others, too) and not be redacted in their Email Feedback Reports. I have some patches and files to this end, Which I've applied successfully to Mailman 2.1.15 and the suggestion was made on the Users list that I submit them for possible application to mm 3, which would probably be a more appropriate place for this feature. To do this I need to set up a test installation of mm 3 which actually works, and delivers mail to a test list. I don't need to deploy it for general use. -- Lindsay Haisley | "Real programmers use butterflies" FMP Computer Services | 512-259-1190 | - xkcd http://www.fmp.com | From barry at list.org Thu Jun 21 17:32:40 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 21 Jun 2012 11:32:40 -0400 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340288727.16746.24.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> <87hau59mpa.fsf@uwakimon.sk.tsukuba.ac.jp> <1340288727.16746.24.camel@pudina.fmp.com> Message-ID: <20120621113240.793017eb@limelight.wooz.org> On Jun 21, 2012, at 09:25 AM, Lindsay Haisley wrote: >Jessy Kate Schingler said: >> FWIW, i found installing within a virtualenv to be a sanity-inducing >> operation for very similar reasons. >> >What's involved here? It's really pretty simple. I'm not sure what the default version of Python is for Mint (it's 2.7 for Ubuntu 12.04), but either 2.6 or 2.7 should work fine, and it's a bug if it doesn't. Anyway, your steps should be something along the lines of: * apt-get install python-virtualenv * virtualenv /path/to/your/mm3/installation * source /path/to/your/mm3/installation/bin/activate * python setup.py install At this point, you actually don't need to be in your virtualenv (or really for the setup.py install step either) but it can be useful because the bin directory will be put on your $PATH. In any case, what I'd do next is: * edit /path/to/your/mm3/installation/etc/mailman.cfg * mailman start I forget whether `mailman start` needs to be passed the -C option, but I think not. >Jessy Kate, can you send me an online reference on setting up a proper >virtual environment? This would be the perfect thing for a code contribution! Would either your or Jessy like to submit a bug and some improved deployment documentation, either as a branch, patch, or .rst file? Cheers, -Barry From barry at list.org Thu Jun 21 17:43:28 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 21 Jun 2012 11:43:28 -0400 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340292745.16746.32.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> <1340292745.16746.32.camel@pudina.fmp.com> Message-ID: <20120621114328.4adcb1f8@limelight.wooz.org> On Jun 21, 2012, at 10:32 AM, Lindsay Haisley wrote: >Barry, I don't know if you followed the discussion on the Users list >regarding placing an encrypted (AES) header into personalized, VERPed >list posts which would contain the subscriber recipient's address and >possibly also the list address. This would satisfy AOL's TOS (others, >too) and not be redacted in their Email Feedback Reports. I've been skimming it, so I think I get the gist. >I have some patches and files to this end, Which I've applied >successfully to Mailman 2.1.15 and the suggestion was made on the Users >list that I submit them for possible application to mm 3, which would >probably be a more appropriate place for this feature. Agreed. I think *something* like this would be useful for mm3. I'm not sure about the best way to do it, i.e. whether it should be a generally available feature enabled by some config, or whether it should be implemented as a plugin. We could certainly discuss all that here. >To do this I need to set up a test installation of mm 3 which actually >works, and delivers mail to a test list. I don't need to deploy it for >general use. Cool. The previous virtualenv instructions should still be relevant. You'd of course need to hook it up to an MTA, but mm3 has a nice "development" mode you can enable, which will ensure that the SMTP RCPT TO header will be forced to a specific address. What this means is that with devmode enabled, you can forge all the RFC 5322 headers you want to make it look like real email, and it'll never escape your local test environment. E.g. if you put the following in your mailman.cfg file: [devmode] enabled: yes recipient: me at example.com *All* of the email sent through your mm3 deployment will actually get delivered to me at example.com. There are other ways to do similar things e.g. using Postfix transports, but you may not need them. To make something like this a supported feature does require more work, e.g. probably a schema change, plumbing through REST, documentation and a test suite. A Launchpad bug and merge proposal will be ideal. But don't let that discourage you. If you can come up with a nice proof-of-concept once of us can help shepherd you through the rest of the process. Cheers, -Barry From fmouse-mailman at fmp.com Thu Jun 21 18:02:04 2012 From: fmouse-mailman at fmp.com (Lindsay Haisley) Date: Thu, 21 Jun 2012 11:02:04 -0500 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <20120621114328.4adcb1f8@limelight.wooz.org> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> <1340292745.16746.32.camel@pudina.fmp.com> <20120621114328.4adcb1f8@limelight.wooz.org> Message-ID: <1340294524.16746.48.camel@pudina.fmp.com> On Thu, 2012-06-21 at 11:43 -0400, Barry Warsaw wrote: > On Jun 21, 2012, at 10:32 AM, Lindsay Haisley wrote: >>I have some patches and files to this end, Which I've applied > >successfully to Mailman 2.1.15 and the suggestion was made on the Users > >list that I submit them for possible application to mm 3, which would > >probably be a more appropriate place for this feature. > > Agreed. I think *something* like this would be useful for mm3. I'm not sure > about the best way to do it, i.e. whether it should be a generally available > feature enabled by some config, or whether it should be implemented as a > plugin. We could certainly discuss all that here. The way I switched it on in 2.1.15 was by way of configuration variable in mm_cfg.py containing a global AES secret key. This can be added to mm_cfg.py by running a key generation script in the runtime mm bin directory. Absent this variable, the feature is moot since all relevant code gets bypassed. > >To do this I need to set up a test installation of mm 3 which actually > >works, and delivers mail to a test list. I don't need to deploy it for > >general use. > > To make something like this a supported feature does require more work, > e.g. probably a schema change, plumbing through REST, documentation and a test > suite. A Launchpad bug and merge proposal will be ideal. But don't let that > discourage you. If you can come up with a nice proof-of-concept once of us > can help shepherd you through the rest of the process. Great!! Something to do with all my FREE TIME . I'll probably work on it since I can't seem to leave projects like this alone. One caveat/issue with the work I've done so far is that it requires the python-crypto module. This is no problem on Linux, possibly not on other Unix-like OSes, but from what I've read, the module has longstanding issues on a Windows platform. This might be a show-stopper. Thanks for the coaching on setting up virtualenv! -- Lindsay Haisley |"Friends are like potatoes. FMP Computer Services | If you eat them, they die" 512-259-1190 | http://www.fmp.com | - Aaron Edmund From barry at list.org Thu Jun 21 18:20:08 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 21 Jun 2012 12:20:08 -0400 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340294524.16746.48.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> <1340292745.16746.32.camel@pudina.fmp.com> <20120621114328.4adcb1f8@limelight.wooz.org> <1340294524.16746.48.camel@pudina.fmp.com> Message-ID: <20120621122008.5091b071@limelight.wooz.org> On Jun 21, 2012, at 11:02 AM, Lindsay Haisley wrote: >The way I switched it on in 2.1.15 was by way of configuration variable >in mm_cfg.py containing a global AES secret key. This can be added to >mm_cfg.py by running a key generation script in the runtime mm bin >directory. Absent this variable, the feature is moot since all relevant >code gets bypassed. So the mm3 equivalent would probably be a new variable in the [mta] section of the mailman.cfg file. Note that the defaults for mailman.cfg live in the src/mailman/config/schema.cfg file, so that's where you'd add this. Note too that these control global system-wide configurations, so if there's any reason why you'd want to have different keys for different mailing lists, it can't go there (but I don't see why that would be so I won't describe the alternative). Also, of course the key will live in plain text in the deployed mailman.cfg file, but it sounds like that's not a problem. >One caveat/issue with the work I've done so far is that it requires the >python-crypto module. This is no problem on Linux, possibly not on other >Unix-like OSes, but from what I've read, the module has longstanding issues >on a Windows platform. This might be a show-stopper. Nope, Mailman doesn't officially run on Windows. :) OTOH, I would make this feature conditional on the module being importable, so it won't be required even on *nix systems. >Thanks for the coaching on setting up virtualenv! No problem, good luck! Don't forget too, that if you can do IRC and need more real-time feedback, I usually hang out on freenode's #mailman channel during US/Eastern working hours. You need to ping my nick (the always clever and hard to guess 'barry' :) to get my attention though. -Barry From fmouse at fmp.com Thu Jun 21 19:00:21 2012 From: fmouse at fmp.com (Lindsay Haisley) Date: Thu, 21 Jun 2012 12:00:21 -0500 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <20120621122008.5091b071@limelight.wooz.org> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> <1340292745.16746.32.camel@pudina.fmp.com> <20120621114328.4adcb1f8@limelight.wooz.org> <1340294524.16746.48.camel@pudina.fmp.com> <20120621122008.5091b071@limelight.wooz.org> Message-ID: <1340298021.644.8.camel@pudina.fmp.com> On Thu, 2012-06-21 at 12:20 -0400, Barry Warsaw wrote: > Note too > that these control global system-wide configurations, so if there's any reason > why you'd want to have different keys for different mailing lists, it can't go > there (but I don't see why that would be so I won't describe the alternative). I gave this some thought and decided that since any automated expunging of subscribers probably should be done at a system level (although theoretically a list admin could script an an automated mail command exchange system, or automate an interaction with a web UI), that a global key would be fine. Perhaps the feature could be turned on or off in individual lists. > Also, of course the key will live in plain text in the deployed mailman.cfg > file, but it sounds like that's not a problem. This isn't for *real* security, just to provide an encryption envelope for information so as to reversibly obfuscate it. -- Lindsay Haisley | "Everything works if you let it" FMP Computer Services | 512-259-1190 | --- The Roadie http://www.fmp.com | From barry at list.org Thu Jun 21 19:44:23 2012 From: barry at list.org (Barry Warsaw) Date: Thu, 21 Jun 2012 13:44:23 -0400 Subject: [Mailman-Developers] Trying to install mm 3.0.0b1 In-Reply-To: <1340298021.644.8.camel@pudina.fmp.com> References: <1340231862.24398.10.camel@pudina.fmp.com> <20120621103609.0a9fb13d@limelight.wooz.org> <1340292745.16746.32.camel@pudina.fmp.com> <20120621114328.4adcb1f8@limelight.wooz.org> <1340294524.16746.48.camel@pudina.fmp.com> <20120621122008.5091b071@limelight.wooz.org> <1340298021.644.8.camel@pudina.fmp.com> Message-ID: <20120621134423.6e8a6eee@limelight.wooz.org> On Jun 21, 2012, at 12:00 PM, Lindsay Haisley wrote: >I gave this some thought and decided that since any automated expunging >of subscribers probably should be done at a system level (although >theoretically a list admin could script an an automated mail command >exchange system, or automate an interaction with a web UI), that a >global key would be fine. Perhaps the feature could be turned on or off >in individual lists. > >> Also, of course the key will live in plain text in the deployed mailman.cfg >> file, but it sounds like that's not a problem. > >This isn't for *real* security, just to provide an encryption envelope >for information so as to reversibly obfuscate it. Both sounds good. Thanks for the info. -Barry From danci_emanuel at yahoo.com Mon Jun 25 13:51:46 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Mon, 25 Jun 2012 04:51:46 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations Message-ID: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> Hi! My name is Emanuel Danci and I am the GSoC student working on integrating?the Systers` feature, dynamic sublists -?http://systers.org/systers-dev/doku.php/good_to_know?s%5B%5D=postgresql?, with Mm 3.0 and I have 2 questions in regard to how the database works. First of all, how is the aliases set populated in time? Second of all, when a user is unsubscribed, is all the data related to him/her deleted (his/hers preferences, the info related to the address etc) ?? ? Thank you, Emanuel? From barry at list.org Mon Jun 25 15:42:58 2012 From: barry at list.org (Barry Warsaw) Date: Mon, 25 Jun 2012 09:42:58 -0400 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> Message-ID: <20120625094258.174b431f@limelight.wooz.org> Hi Emanuel, On Jun 25, 2012, at 04:51 AM, Danci Emanuel wrote: >First of all, how is the aliases set populated in time? I'm guessing you mean the mailing list's "acceptable aliases", i.e. addresses which are not the list's posting address, but will still be accepted if it appears in a recipient header (To or CC) instead of being rejected as "implicitly addressed". At the lowest level, you adapt a mailing list into an IAcceptableAliasSet, and then you .add() or .remove() addresses from that object. Of course, all this is plumbed through the REST API. A good description of how this works is available here: http://packages.python.org/mailman/src/mailman/rest/docs/configuration.html#acceptable-aliases >Second of all, when a user is unsubscribed, is all the data related to >him/her deleted (his/hers preferences, the info related to the address etc) >? In order to understand what happens when a user is unsubscribed, it's important to review the data model. Mailman has "addresses", "users", and "members". An address is essentially a known email address, which may be verified or not, and which may or may not be linked to a user. Users aren't much more than a user id, but may have various other pieces of information associated with it (e.g. a password). Both addresses and users may have "display names" and preferences. Neither of these objects are directly associated with a mailing list. Addresses can be subscribed to mailing lists. This "subscription" is represented by member objects. Or put another way, a member is an address associated with a mailing list with a given role. Because addresses can be linked to users, this is how a user gets subscribed to a mailing list. Note that if a user has a "preferred address", the user record can be associated with the mailing list as well, but this is a convenience shortcut. Members also have preferences. So what happens when someone unsubscribes from a mailing list? All that happens is that the relevant member record gets deleted. The member's preferences also get deleted, but that's it. Neither the address nor the user is touched. Hope that helps! -Barry From danci_emanuel at yahoo.com Mon Jun 25 17:02:10 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Mon, 25 Jun 2012 08:02:10 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120625094258.174b431f@limelight.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> Message-ID: <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> Thanks for the clarifications, Barry! I still want to ask you something. As far as I am aware, I also have to implement the aliases feature as Systers see it, meaning that a user, after gets registered can post messages to a mailing list from more than one address. So, just to make sure, do you think that these addresses could be added directly to the address table and? then all the addresses (aliases) linked to the same user?? This -?http://systers.org/systers-dev/doku.php/systers_database_in_postgresql?s[]=database? is the current database schema implemented by Systes, so basically I am looking a solution? to replace the 'alias' table. Furthermore, in regard to the user removal, if the address and the user records remain in the database, how can we know if a user has been unsubscribed from the list? I am asking this, because in order to successfully implement the dlists, we need to keep account of the unsubscribed users, so that we won`t send any unwanted emails? to them. In the aforementioned Systers` schema, this is done by setting the 'deleted' flag to true. Thanks again, Emanuel ________________________________ From: Barry Warsaw To: mailman-developers at python.org Cc: Danci Emanuel Sent: Monday, June 25, 2012 4:42 PM Subject: Re: [Mailman-Developers] Questions in regard to the database operations Hi Emanuel, On Jun 25, 2012, at 04:51 AM, Danci Emanuel wrote: >First of all, how is the aliases set populated in time? I'm guessing you mean the mailing list's "acceptable aliases", i.e. addresses which are not the list's posting address, but will still be accepted if it appears in a recipient header (To or CC) instead of being rejected as "implicitly addressed". At the lowest level, you adapt a mailing list into an IAcceptableAliasSet, and then you .add() or .remove() addresses from that object. Of course, all this is plumbed through the REST API.? A good description of how this works is available here: http://packages.python.org/mailman/src/mailman/rest/docs/configuration.html#acceptable-aliases >Second of all, when a user is unsubscribed, is all the data related to >him/her deleted (his/hers preferences, the info related to the address etc) >? In order to understand what happens when a user is unsubscribed, it's important to review the data model.? Mailman has "addresses", "users", and "members".? An address is essentially a known email address, which may be verified or not, and which may or may not be linked to a user.? Users aren't much more than a user id, but may have various other pieces of information associated with it (e.g. a password).? Both addresses and users may have "display names" and preferences.? Neither of these objects are directly associated with a mailing list. Addresses can be subscribed to mailing lists.? This "subscription" is represented by member objects.? Or put another way, a member is an address associated with a mailing list with a given role.? Because addresses can be linked to users, this is how a user gets subscribed to a mailing list.? Note that if a user has a "preferred address", the user record can be associated with the mailing list as well, but this is a convenience shortcut.? Members also have preferences. So what happens when someone unsubscribes from a mailing list?? All that happens is that the relevant member record gets deleted.? The member's preferences also get deleted, but that's it.? Neither the address nor the user is touched. Hope that helps! -Barry From barry at list.org Mon Jun 25 20:39:51 2012 From: barry at list.org (Barry Warsaw) Date: Mon, 25 Jun 2012 14:39:51 -0400 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> Message-ID: <20120625143951.7ebdc02a@limelight.wooz.org> On Jun 25, 2012, at 08:02 AM, Danci Emanuel wrote: >I still want to ask you something. As far as I am aware, I also have to >implement the aliases feature as Systers see it, meaning that a user, after >gets registered can post messages to a mailing list from more than one >address. Ah, so this is totally different than "acceptable aliases" and the IAcceptableAliasSet interface. What I described earlier is a mailing list feature which lets you set up a posting address alias in your MTA. For example, let's say you create a mailing list for your security team and you call it security-team at example.com. For convenience, you want to allow people to post to security at example.com and so you set up an alias in your MTA's configuration file (e.g. /etc/aliases) which points security -> security-team. When people post to security at example.com, it gets forwarded to security-team at example.com, but Mailman will not see the list's posting address in the To header. Without security at example.com being added to the mailing list's acceptable_aliases, these posts will be rejected with an "implicit destination" error. But none of that really applies to your use case. ;) >So, just to make sure, do you think that these addresses could be added >directly to the address table and? then all the addresses (aliases) linked to >the same user?? This >-?http://systers.org/systers-dev/doku.php/systers_database_in_postgresql?s[]=database? >is the current database schema implemented by Systes, so basically I am >looking a solution? to replace the 'alias' table. Remember the model I described earlier, regarding addresses, users, and members? Using this model, you can register multiple address objects and link them to the same user. E.g. you could register all of the following: anne at example.com aperson at example.com anne.x.person at example.com (Let's assume too that all three have been validated.) and link those addresses to the user record representing Anne Person. When Anne subscribes to a mailing list, such as security-team at example.com, she chooses one of those three addresses to subscribe using the 'member' role. This is the address that will receive postings to the list. So if she subscribes anne at example.com, then when new messages are posted to the mailing list, only anne at example.com will receive copies. However, Anne can *post* to the mailing list with any of her three validated and linked email addresses, and no further moderation approval is necessary. This is one of the big advantages of the mm3 model over the mm2 model. We now *know* that all three of those addresses are associated with Anne, so any of her addresses are allowed for posting. However, she will only receive copies of list posts to the addresses she's explicitly subscribed, IOW she won't receive three copies of list postings unless she wants to. In summary, the mm3 model directly supports the Systers model, as I understand it. All the internal APIs are there to add additional addresses, verify them, link them to users, and subscribe them to mailing lists. All you have to work out is the social and procedural ways of getting those additional addresses into the system, verified, and linked to the user. >Furthermore, in regard to the user removal, if the address and the user >records remain in the database, how can we know if a user has been >unsubscribed from the list? You know because there will be no member record linking any of the user's addresses to the mailing list. There are various ways using the internal API to check this. Probably the easiest is to use something like the following: from zope.component import getUtility from mailman.interfaces.member import MemberRole from mailman.interfaces.subscriptions import ISubscriptionService from mailman.interfaces.usermanager import IUserManager service = getUtility(ISubscriptionService) manager = getUtility(IUserManager) # Find the user record for the user linked to this email address. anne = manager.get_user('anne at example.com') if anne is None: # not found # Search for all of Anne's subscriptions on the security team, but ignore # any where she is a list admin or moderator. for member in service.find_members(anne.user_id, 'security-team at example.com', MemberRole.member): # Iterate over all of Anne's memberships on this mailing list. >I am asking this, because in order to successfully implement the dlists, we >need to keep account of the unsubscribed users, so that we won`t send any >unwanted emails? to them. In the aforementioned Systers` schema, this is done >by setting the 'deleted' flag to true. Although you can easily tell whether someone is subscribed to a mailing list or not, you can't really tell if they were once subscribed but now are not. mm3 just doesn't keep a record of that. Now, there are a couple of ways we could allow a plugin to know that. One thing to do is to add an event which gets triggered when an unsubscription occurs. Then your plugin could register interest in that event and record it in whatever local database it uses. That would be pretty easy to add. Another way to do it would be to not actually unsubscribe the member, but instead to disable delivery of the member. It would have the same effect of inhibiting delivery of list posts, but it would keep the member record in the database. Each IMember has a "delivery status" flag which contains an enum value indicating whether delivery is enabled or disabled (the latter with various possible reasons). If you used this approach, you'd have to amend the .find_members() code above to check member.delivery_status. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From danci_emanuel at yahoo.com Tue Jun 26 00:30:33 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Mon, 25 Jun 2012 15:30:33 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120625143951.7ebdc02a@limelight.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> Message-ID: <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> Thank you very much for the clarifications, Barry! Now everything is clear! In regard to the aliases, based on you suggestions I will find a proper way? to link the addresses (aliases) to?a certain user hence solving the problem elegantly with the already existing 'resources'. As for the latter problem, I still have something in mind. If we were to choose the? second option (the one which implies setting the flag "delivery_status" to the proper value) does this not mean that we will have to keep the user`s preferences alive, without deleting?them? I am thinking about a solution to avoid keeping the user`s preferences,?but please correct me if?I am wrong. Could not?we add a flag to the? 'user' table?(e.g: unsubscribed, deleted),?flag that will hold the value 1 if?the user is unsubscribed and 0 otherwise.?Although your solution will work?as a wonder, if I? got everything?wright, adding the flag to the 'user' table will save us some memory.? Do you think I should add that flag to the 'user'?table,?or should I just adapt the code so? that the "delivery_status" will be properly?changed when a user is unsubscribed?? ? Thanks again, Emanuel? ________________________________ From: Barry Warsaw To: Danci Emanuel Cc: mailman-developers at python.org Sent: Monday, June 25, 2012 9:39 PM Subject: Re: [Mailman-Developers] Questions in regard to the database operations On Jun 25, 2012, at 08:02 AM, Danci Emanuel wrote: >I still want to ask you something. As far as I am aware, I also have to >implement the aliases feature as Systers see it, meaning that a user, after >gets registered can post messages to a mailing list from more than one >address. Ah, so this is totally different than "acceptable aliases" and the IAcceptableAliasSet interface.? What I described earlier is a mailing list feature which lets you set up a posting address alias in your MTA. For example, let's say you create a mailing list for your security team and you call it security-team at example.com.? For convenience, you want to allow people to post to security at example.com and so you set up an alias in your MTA's configuration file (e.g. /etc/aliases) which points security -> security-team. When people post to security at example.com, it gets forwarded to security-team at example.com, but Mailman will not see the list's posting address in the To header.? Without security at example.com being added to the mailing list's acceptable_aliases, these posts will be rejected with an "implicit destination" error. But none of that really applies to your use case. ;) >So, just to make sure, do you think that these addresses could be added >directly to the address table and? then all the addresses (aliases) linked to >the same user??? This >-?http://systers.org/systers-dev/doku.php/systers_database_in_postgresql?s[]=database? >is the current database schema implemented by Systes, so basically I am >looking a solution? to replace the 'alias' table. Remember the model I described earlier, regarding addresses, users, and members?? Using this model, you can register multiple address objects and link them to the same user.? E.g. you could register all of the following: anne at example.com aperson at example.com anne.x.person at example.com (Let's assume too that all three have been validated.) and link those addresses to the user record representing Anne Person.? When Anne subscribes to a mailing list, such as security-team at example.com, she chooses one of those three addresses to subscribe using the 'member' role. This is the address that will receive postings to the list. So if she subscribes anne at example.com, then when new messages are posted to the mailing list, only anne at example.com will receive copies. However, Anne can *post* to the mailing list with any of her three validated and linked email addresses, and no further moderation approval is necessary. This is one of the big advantages of the mm3 model over the mm2 model.? We now *know* that all three of those addresses are associated with Anne, so any of her addresses are allowed for posting.? However, she will only receive copies of list posts to the addresses she's explicitly subscribed, IOW she won't receive three copies of list postings unless she wants to. In summary, the mm3 model directly supports the Systers model, as I understand it.? All the internal APIs are there to add additional addresses, verify them, link them to users, and subscribe them to mailing lists.? All you have to work out is the social and procedural ways of getting those additional addresses into the system, verified, and linked to the user. >Furthermore, in regard to the user removal, if the address and the user >records remain in the database, how can we know if a user has been >unsubscribed from the list? You know because there will be no member record linking any of the user's addresses to the mailing list.? There are various ways using the internal API to check this.? Probably the easiest is to use something like the following: from zope.component import getUtility from mailman.interfaces.member import MemberRole from mailman.interfaces.subscriptions import ISubscriptionService from mailman.interfaces.usermanager import IUserManager service = getUtility(ISubscriptionService) manager = getUtility(IUserManager) # Find the user record for the user linked to this email address. anne = manager.get_user('anne at example.com') if anne is None: ? ? # not found # Search for all of Anne's subscriptions on the security team, but ignore # any where she is a list admin or moderator. for member in service.find_members(anne.user_id, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 'security-team at example.com', ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? MemberRole.member): ? ? # Iterate over all of Anne's memberships on this mailing list. >I am asking this, because in order to successfully implement the dlists, we >need to keep account of the unsubscribed users, so that we won`t send any >unwanted emails? to them. In the aforementioned Systers` schema, this is done >by setting the 'deleted' flag to true. Although you can easily tell whether someone is subscribed to a mailing list or not, you can't really tell if they were once subscribed but now are not. mm3 just doesn't keep a record of that. Now, there are a couple of ways we could allow a plugin to know that.? One thing to do is to add an event which gets triggered when an unsubscription occurs.? Then your plugin could register interest in that event and record it in whatever local database it uses.? That would be pretty easy to add. Another way to do it would be to not actually unsubscribe the member, but instead to disable delivery of the member.? It would have the same effect of inhibiting delivery of list posts, but it would keep the member record in the database.? Each IMember has a "delivery status" flag which contains an enum value indicating whether delivery is enabled or disabled (the latter with various possible reasons). If you used this approach, you'd have to amend the .find_members() code above to check member.delivery_status. Cheers, -Barry From barry at list.org Tue Jun 26 02:54:31 2012 From: barry at list.org (Barry Warsaw) Date: Mon, 25 Jun 2012 20:54:31 -0400 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> Message-ID: <20120625205431.3f261d99@resist.wooz.org> On Jun 25, 2012, at 03:30 PM, Danci Emanuel wrote: >As for the latter problem, I still have something in mind. If we were to >choose the? second option (the one which implies setting the flag >"delivery_status" to the proper value) does this not mean that we will have >to keep the user`s preferences alive, without deleting?them? Here's how preferences work. There is an IPreferences interface which describes the kind of things that are "preferences". Members, users, and addresses all can have a pointer to a preferences record. There are also some system default preference values. When we look up a preference on a member, the search order goes like this: * member * address * user * system meaning, if the member doesn't have a preference set, we fall back to the subscribed address, then to the user linked to that address, and finally the system default. At each level, the object only has a preference record if explicitly created. Usually, they don't have preferences (meaning the system default takes over). When a member record is deleted, only that member's preferences are deleted. If either the address and user associated with that member has a preferences record, it *has* to stick around because an address or user can be subscribed to many mailing lists. E.g. You could decide that you want all your postings to be acknowledged. You'd set that on your user record and then all your subscriptions would automatically inherit it. I don't see much savings in trying not to delete a member's preferences when being unsubscribed. It's just a row in the database, and probably not a very big one at that. Plus, it'll usually be empty anyway. Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From p at state-of-mind.de Tue Jun 26 13:51:30 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 26 Jun 2012 13:51:30 +0200 Subject: [Mailman-Developers] [GSoC 2012] Metrics In-Reply-To: <20120517210928.GB19285@state-of-mind.de> References: <20120505162628.4fed030f@limelight.wooz.org> <20120517210928.GB19285@state-of-mind.de> Message-ID: <20120626115129.GP22956@state-of-mind.de> * Patrick Ben Koetter

: > let me throw in some thoughts just to annoy you ;) > > Like with most statistical data I mostly see the figures being used to give > statements on quantity - top poster, number of threads etc. Do you think it > would be possible to also make some statements on quality? > > Let me give an example: Mailing lists are often places where people go to ask > for advice. Someone asking usually starts a thread and continually keeps > replying. That easily makes a person top poster and might make the same person > a thread starter, but number of posts and threads started gives no indication > of that persons knowledge (concerning the mailing lists topic). > > OTOH someone who has been on the list for ages, who replies more often than > starting threads and who ends threads often after she has replied might very > well be a very knowledgeable person, because she gives the one answer that > solves the problem. > > Do you think it would be possible to deduct such quality oriented statements? As a follow-up: I just stumbled across http://www.mentby.com/patrick-ben-koetter/, which is nice because it also gives an overview over all (here: some) mailing lists an identity posts to. The second pie chart seems to try to say something about quality. It splits posts in 'relevant' and 'passive', which are not exactly opposites, but well ? Actually I'd say they still need to work on their rating: ;) p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From danci_emanuel at yahoo.com Wed Jun 27 01:43:56 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Tue, 26 Jun 2012 16:43:56 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120625205431.3f261d99@resist.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> Message-ID: <1340754236.53165.YahooMailNeo@web110415.mail.gq1.yahoo.com> Thanks for all the explanations, Barry! Now everything is much more clear! Indeed, is not necessary for us to save every?bit of space, so I will? go with the second option that you presented initially, by?setting the? "delivery_status" flag to the appropriate value. Thanks again for you help! ? Emanuel? ________________________________ From: Barry Warsaw To: Danci Emanuel Cc: "mailman-developers at python.org" Sent: Tuesday, June 26, 2012 3:54 AM Subject: Re: [Mailman-Developers] Questions in regard to the database operations On Jun 25, 2012, at 03:30 PM, Danci Emanuel wrote: >As for the latter problem, I still have something in mind. If we were to >choose the? second option (the one which implies setting the flag >"delivery_status" to the proper value) does this not mean that we will have >to keep the user`s preferences alive, without deleting?them? Here's how preferences work. There is an IPreferences interface which describes the kind of things that are "preferences".? Members, users, and addresses all can have a pointer to a preferences record.? There are also some system default preference values. When we look up a preference on a member, the search order goes like this: * member * address * user * system meaning, if the member doesn't have a preference set, we fall back to the subscribed address, then to the user linked to that address, and finally the system default.? At each level, the object only has a preference record if explicitly created.? Usually, they don't have preferences (meaning the system default takes over). When a member record is deleted, only that member's preferences are deleted. If either the address and user associated with that member has a preferences record, it *has* to stick around because an address or user can be subscribed to many mailing lists.? E.g. You could decide that you want all your postings to be acknowledged.? You'd set that on your user record and then all your subscriptions would automatically inherit it. I don't see much savings in trying not to delete a member's preferences when being unsubscribed.? It's just a row in the database, and probably not a very big one at that.? Plus, it'll usually be empty anyway. Cheers, -Barry From iane at sussex.ac.uk Wed Jun 27 12:23:51 2012 From: iane at sussex.ac.uk (Ian Eiloart) Date: Wed, 27 Jun 2012 10:23:51 +0000 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120625205431.3f261d99@resist.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> Message-ID: <63105996-2A3B-409B-87A6-F9DDB18ACADA@sussex.ac.uk> On 26 Jun 2012, at 01:54, Barry Warsaw wrote: > On Jun 25, 2012, at 03:30 PM, Danci Emanuel wrote: > >> As for the latter problem, I still have something in mind. If we were to >> choose the second option (the one which implies setting the flag >> "delivery_status" to the proper value) does this not mean that we will have >> to keep the user`s preferences alive, without deleting them? > > Here's how preferences work. > > There is an IPreferences interface which describes the kind of things that are > "preferences". Members, users, and addresses all can have a pointer to a > preferences record. There are also some system default preference values. > When we look up a preference on a member, the search order goes like this: > > * member > * address > * user > * system Do you mean "membership"? If I (a user) am a member of a list, then I have a membership record. But, the member is the user. If the term "member record" is used to describe a membership record, things are going to get confusing, I think. Or, perhaps it's a subscription (but still not a subscriber) record? To be more explicit, I'd expect a "members" table to simply list references to users. Whereas a "subscriptions" or "memberships" table would include references to members, as well as information about the subscription (preferred mailing address, start date, permission type, subject line munging, etc, etc). My guess is that the link to users is implied, though? > meaning, if the member doesn't have a preference set, we fall back to the > subscribed address, then to the user linked to that address, and finally the > system default. At each level, the object only has a preference record if > explicitly created. Usually, they don't have preferences (meaning the system > default takes over). > > When a member record is deleted, only that member's preferences are deleted. > If either the address and user associated with that member has a preferences > record, it *has* to stick around because an address or user can be subscribed > to many mailing lists. E.g. You could decide that you want all your postings > to be acknowledged. You'd set that on your user record and then all your > subscriptions would automatically inherit it. > > I don't see much savings in trying not to delete a member's preferences when > being unsubscribed. It's just a row in the database, and probably not a very > big one at that. Plus, it'll usually be empty anyway. > > Cheers, > -Barry > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers at python.org > http://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ > Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/iane%40sussex.ac.uk > > Security Policy: http://wiki.list.org/x/QIA9 -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From danci_emanuel at yahoo.com Wed Jun 27 17:19:21 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Wed, 27 Jun 2012 08:19:21 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120625205431.3f261d99@resist.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> Message-ID: <1340810361.58129.YahooMailNeo@web110407.mail.gq1.yahoo.com> Hi Barry! I am back with some questions in regard to the aliases. After a persons becomes a member of a particular list, subscribing? with its primary email address (the address at which he/she? will receive?the replies), how can that person add the additional? email?addresses (aliases) to the database? Thanks,? Emanuel From danci_emanuel at yahoo.com Wed Jun 27 20:07:19 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Wed, 27 Jun 2012 11:07:19 -0700 (PDT) Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 Message-ID: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> Hello everyone! This is the continuation of the topic started on the mailman-users mailing list. Here is the original post:?http://mail.python.org/pipermail/mailman-users/2012-June/073632.html And this is the post at which the conversation stopped:?http://mail.python.org/pipermail/mailman-users/2012-June/073636.html Basically, what I am trying to do is to set up a running version of Mailman on my local machine, by using the 5 minute guide (http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mailman/docs/WebUIin5.rst).? ? After doing what Mark Sapiro suggested in the last post, I got this: Jun 27 20:01:09 2012 (5011) [SMTPServer] listening: localhost:9025 Jun 27 20:01:09 2012 (5011) starting the SMTP server thread Jun 27 20:01:09 2012 (5011) connecting to localhost:9025 Jun 27 20:01:09 2012 (5011) [SMTPServer] starting asyncore loop Traceback (most recent call last): ? File "", line 1, in ? File "/usr/local/mm3/mailman/src/mailman/testing/layers.py", line 254, in setUp ? ? cls.smtpd.start() ? File "/usr/local/mm3/mailman/src/mailman/testing/mta.py", line 197, in start ? ? QueueController.start(self) ? File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 69, in start ? ? smtpd = self._connect() ? File "/usr/local/mm3/mailman/eggs/lazr.smtptest-1.3-py2.6.egg/lazr/smtptest/controller.py", line 60, in _connect ? ? smtpd.connect(self._server.host, self._server.port) ? File "/usr/lib/python2.6/smtplib.py", line 295, in connect ? ? self.sock = self._get_socket(host, port, self.timeout) ? File "/usr/lib/python2.6/smtplib.py", line 273, in _get_socket ? ? return socket.create_connection((port, host), timeout) ? File "/usr/lib/python2.6/socket.py", line 514, in create_connection ? ? raise error, msg error: [Errno 111] Connection refused Also, there is one more thing that could be related to this problem. I also have installed Mailman 2.1.10 and after setting it up for the proper domain, when a user tries to subscribe to one of the lists, although? the subscription?is recorded, the confirmation email is never sent back. The message that appears in? the smtp-failure log?is this: Jun 04 20:55:57 2012 (10427) Low level smtp error: (111, 'Connection refused'), msgid: Jun 04 20:55:57 2012 (10427) delivery to esdanci at acm.ro failed with code -1: (111, 'Connection refused') Thanks, Emanuel? From barry at list.org Wed Jun 27 23:31:39 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 27 Jun 2012 17:31:39 -0400 Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> Message-ID: <20120627173139.7b182b5e@limelight.wooz.org> On Jun 27, 2012, at 11:07 AM, Danci Emanuel wrote: >This is the continuation of the topic started on the mailman-users mailing >list. Thanks for moving the discussion here. >Here is the original >post:?http://mail.python.org/pipermail/mailman-users/2012-June/073632.html >And this is the post at which the conversation >stopped:?http://mail.python.org/pipermail/mailman-users/2012-June/073636.html > > >Basically, what I am trying to do is to set up a running version of Mailman >on my local machine, by using the 5 minute guide >(http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mailman/docs/WebUIin5.rst).? >? > >After doing what Mark Sapiro suggested in the last post, I got this: So, in Mark's suggestion, this exception occurs after you've called x.setUp()? Or does that call succeed? If it does succeed, don't call x.tearDown(), but instead go to a different shell on the same machine and try to `telnet localhost 9025`. What happens then? You could try changing the port that the testing framework uses to list for SMTP (and other) connections. That is set in the file src/mailman/testing/testing.cfg. Does it help if you change the smtp_port value? Do you have any kind of local firewall set up? What OS are you using? Cheers, -Barry From mark at msapiro.net Thu Jun 28 00:17:57 2012 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 27 Jun 2012 15:17:57 -0700 Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> Message-ID: <4FEB8695.1000009@msapiro.net> On 6/27/2012 11:07 AM, Danci Emanuel wrote: > > Also, there is one more thing that could be related to this problem. I also have installed Mailman 2.1.10 > and after setting it up for the proper domain, when a user tries to subscribe to one of the lists, although > the subscription is recorded, the confirmation email is never sent back. The message that appears in > the smtp-failure log is this: > > Jun 04 20:55:57 2012 (10427) Low level smtp error: (111, 'Connection refused'), msgid: > Jun 04 20:55:57 2012 (10427) delivery to esdanci at acm.ro failed with code -1: (111, 'Connection refused') Barry has responded to the MM 3 part of this. With respect to the above, there is no MTA listening on SMTPHOST/SMTPPORT (default licalhost/25) or connection is blocked by a firewall. See the FAQ at . -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From barry at list.org Thu Jun 28 00:43:34 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 27 Jun 2012 18:43:34 -0400 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <1340810361.58129.YahooMailNeo@web110407.mail.gq1.yahoo.com> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> <1340810361.58129.YahooMailNeo@web110407.mail.gq1.yahoo.com> Message-ID: <20120627184334.21c52849@limelight.wooz.org> Hi again Danci. Keep the great questions coming! :) On Jun 27, 2012, at 08:19 AM, Danci Emanuel wrote: >I am back with some questions in regard to the aliases. After a persons >becomes a member of a particular list, subscribing? with its primary email >address (the address at which he/she? will receive?the replies), how can that >person add the additional? email?addresses (aliases) to the database? Yes. Subscriptions (i.e. memberships) are completely separate from addresses, which again are separate from users. Users and addresses can each be added separately at any time. Many of the doctests in src/mailman/model/docs illustrate this. In mm3 terminology, these separately added users and addresses are "unlinked". Usually addresses and users are linked, and an address can point to exactly one user, while users can point to many addresses. Members link a mailing list to either a user or an address, but not both. A user can be linked only if it has a preferred address, in which case message delivery will be to that preferred address. This allows a user to subscribe to many mailing lists with their preferred address, and then change their preferred address in one swoop to change all of their deliveries. So, to answer your second question, the way a user adds additional addresses is fairly simple. Given an IUser object, you can just call the .register() method to both register a new email and to link that to the user. The address will be unverified though, so in order to accept postings from that new email, it has to be verified (which through the internal API is just setting a flag). Does that answer the question? Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From barry at list.org Thu Jun 28 00:57:23 2012 From: barry at list.org (Barry Warsaw) Date: Wed, 27 Jun 2012 18:57:23 -0400 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <63105996-2A3B-409B-87A6-F9DDB18ACADA@sussex.ac.uk> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> <63105996-2A3B-409B-87A6-F9DDB18ACADA@sussex.ac.uk> Message-ID: <20120627185723.1ffd04b7@limelight.wooz.org> On Jun 27, 2012, at 10:23 AM, Ian Eiloart wrote: >> There is an IPreferences interface which describes the kind of things that >> are "preferences". Members, users, and addresses all can have a pointer to >> a preferences record. There are also some system default preference >> values. When we look up a preference on a member, the search order goes >> like this: >> >> * member >> * address >> * user >> * system >Do you mean "membership"? If I (a user) am a member of a list, then I have a >membership record. It means that you-the-user or you-an-address-you-control is linked to the mailing list through the member record. >But, the member is the user. Not really. It's perfectly valid for a member record to link a mailing list to an address which is not associated with any user record. Members link addresses to mailing lists. The shortcut of linking a user to a mailing list is only valid if that user has a preferred address. >If the term "member record" is used to describe a membership record, things >are going to get confusing, I think. Or, perhaps it's a subscription (but >still not a subscriber) record? Member records represent subscriptions to mailing lists as a recipient of messages sent to the list. But they also represent other mailing list roles, such as list-owners and list-moderators. So let's say anne at example.com is a member of a mailing list, and the list owner. Her address record will be linked to two member records, one with role "member" and the other with role "owner". This might seem confusing at first, but I've lived with this terminology for a long time, after many clarifying steps, and I think this works the best. Once you understand the relationship of users, addresses, members, and mailing lists, it shouldn't be confusing. >To be more explicit, I'd expect a "members" table to simply list references >to users. It can't because of the types of relationships we want to support, such as: - A user subscribing multiple addresses to a mailing list. - A user serving more than one role for a mailing list. - Subscribing non-user addresses to mailing list, e.g. mailbots - Subscribing a user's preferred address, which can easily be changed system-wide. Memberships link addresses to mailing lists. They *can* link a user to a mailing list, but only if that user has a preferred address. >Whereas a "subscriptions" or "memberships" table would include references to >members, as well as information about the subscription (preferred mailing >address, start date, permission type, subject line munging, etc, etc). I'm not sure what a "member" would be if not essentially equivalent to a subscription. IOW, when you say "expect a members table to simply list references to users", I don't see how that's useful as an independent concept. >My guess is that the link to users is implied, though? In the case where a member links an address to a mailing list, and that address is linked to a user, then you're correct. :). Through queries (hidden behind other interfaces, such as the IRoster) we can always find the list of users that are subscribed to a mailing list, although of course user-less addresses won't show up there. Stephen did a pretty good job of explaining all this very succinctly. A diagram would help, but I suck at ascii art. ;) http://packages.python.org/mailman/src/mailman/docs/8-miles-high.html#user-model Contributions welcome of course! Cheers, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: not available URL: From iane at sussex.ac.uk Thu Jun 28 13:24:26 2012 From: iane at sussex.ac.uk (Ian Eiloart) Date: Thu, 28 Jun 2012 11:24:26 +0000 Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120627185723.1ffd04b7@limelight.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> <63105996-2A3B-409B-87A6-F9DDB18ACADA@sussex.ac.uk> <20120627185723.1ffd04b7@limelight.wooz.org> Message-ID: <85C53B79-1A63-4074-9A72-7F89D993AAE7@sussex.ac.uk> On 27 Jun 2012, at 23:57, Barry Warsaw wrote: > On Jun 27, 2012, at 10:23 AM, Ian Eiloart wrote: > >>> There is an IPreferences interface which describes the kind of things that >>> are "preferences". Members, users, and addresses all can have a pointer to >>> a preferences record. There are also some system default preference >>> values. When we look up a preference on a member, the search order goes >>> like this: >>> >>> * member >>> * address >>> * user >>> * system > >> Do you mean "membership"? If I (a user) am a member of a list, then I have a >> membership record. > > It means that you-the-user or you-an-address-you-control is linked to the > mailing list through the member record. I guess it's too late for me to be talking about this because it's all coded. But, if there's a choice in the UI, I just think that in the real world, we'd talk about "members" of an organisation being people, or other organisations. It might be that the only information you have about a member is their email address, so that works too. Then we'd talk about "membership" or "subscription" records, which would tell you who the member was, and various things about their membership (like when it started, communication preferences, and so on). So, I just think "membership" would be a more natural description of that record, and therefore slightly easier for newcomers to understand. >> But, the member is the user. > > Not really. It's perfectly valid for a member record to link a mailing list > to an address which is not associated with any user record. Members link > addresses to mailing lists. The shortcut of linking a user to a mailing list > is only valid if that user has a preferred address. > >> If the term "member record" is used to describe a membership record, things >> are going to get confusing, I think. Or, perhaps it's a subscription (but >> still not a subscriber) record? > > Member records represent subscriptions to mailing lists as a recipient of > messages sent to the list. But they also represent other mailing list roles, > such as list-owners and list-moderators. So let's say anne at example.com is a > member of a mailing list, and the list owner. Her address record will be > linked to two member records, one with role "member" and the other with role > "owner". > > This might seem confusing at first, but I've lived with this terminology for a > long time, after many clarifying steps, and I think this works the best. Once > you understand the relationship of users, addresses, members, and mailing > lists, it shouldn't be confusing. > >> To be more explicit, I'd expect a "members" table to simply list references >> to users. > > It can't because of the types of relationships we want to support, such as: > > - A user subscribing multiple addresses to a mailing list. > - A user serving more than one role for a mailing list. > - Subscribing non-user addresses to mailing list, e.g. mailbots > - Subscribing a user's preferred address, which can easily be changed > system-wide. > > Memberships link addresses to mailing lists. They *can* link a user to a > mailing list, but only if that user has a preferred address. > >> Whereas a "subscriptions" or "memberships" table would include references to >> members, as well as information about the subscription (preferred mailing >> address, start date, permission type, subject line munging, etc, etc). > > I'm not sure what a "member" would be if not essentially equivalent to a > subscription. IOW, when you say "expect a members table to simply list > references to users", I don't see how that's useful as an independent > concept. > >> My guess is that the link to users is implied, though? > > In the case where a member links an address to a mailing list, and that > address is linked to a user, then you're correct. :). > > Through queries (hidden behind other interfaces, such as the IRoster) we can > always find the list of users that are subscribed to a mailing list, although > of course user-less addresses won't show up there. > > Stephen did a pretty good job of explaining all this very succinctly. A > diagram would help, but I suck at ascii art. ;) > > http://packages.python.org/mailman/src/mailman/docs/8-miles-high.html#user-model > > Contributions welcome of course! > > Cheers, > -Barry -- Ian Eiloart Postmaster, University of Sussex +44 (0) 1273 87-3148 From danci_emanuel at yahoo.com Fri Jun 29 01:17:18 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Thu, 28 Jun 2012 16:17:18 -0700 (PDT) Subject: [Mailman-Developers] Questions in regard to the database operations In-Reply-To: <20120627184334.21c52849@limelight.wooz.org> References: <1340625106.63640.YahooMailNeo@web110401.mail.gq1.yahoo.com> <20120625094258.174b431f@limelight.wooz.org> <1340636530.41762.YahooMailNeo@web110410.mail.gq1.yahoo.com> <20120625143951.7ebdc02a@limelight.wooz.org> <1340663433.84997.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120625205431.3f261d99@resist.wooz.org> <1340810361.58129.YahooMailNeo@web110407.mail.gq1.yahoo.com> <20120627184334.21c52849@limelight.wooz.org> Message-ID: <1340925438.39523.YahooMailNeo@web110412.mail.gq1.yahoo.com> Hi Barry! Yes, in terms of how the back-end works, it answers it. But, I am interested also in the following aspect. Lets say that a users subscribes to a list with his/her preferred? address. Afterwards, how can that user, who does not have access to the back end? of the mailing list, add other email addresses from which he/she can send emails to the list? This is related strictly how a user would do it, not necessarily how the code works in this case. ?Thanks, Emanuel? ________________________________ From: Barry Warsaw To: Danci Emanuel Cc: "mailman-developers at python.org" Sent: Thursday, June 28, 2012 1:43 AM Subject: Re: [Mailman-Developers] Questions in regard to the database operations Hi again Danci.? Keep the great questions coming! :) On Jun 27, 2012, at 08:19 AM, Danci Emanuel wrote: >I am back with some questions in regard to the aliases.? After a persons >becomes a member of a particular list, subscribing? with its primary email >address (the address at which he/she? will receive?the replies), how can that >person add the additional? email?addresses (aliases) to the database? Yes.? Subscriptions (i.e. memberships) are completely separate from addresses, which again are separate from users.? Users and addresses can each be added separately at any time.? Many of the doctests in src/mailman/model/docs illustrate this.? In mm3 terminology, these separately added users and addresses are "unlinked". Usually addresses and users are linked, and an address can point to exactly one user, while users can point to many addresses. Members link a mailing list to either a user or an address, but not both.? A user can be linked only if it has a preferred address, in which case message delivery will be to that preferred address.? This allows a user to subscribe to many mailing lists with their preferred address, and then change their preferred address in one swoop to change all of their deliveries. So, to answer your second question, the way a user adds additional addresses is fairly simple.? Given an IUser object, you can just call the .register() method to both register a new email and to link that to the user.? The address will be unverified though, so in order to accept postings from that new email, it has to be verified (which through the internal API is just setting a flag). Does that answer the question? Cheers, -Barry From danci_emanuel at yahoo.com Fri Jun 29 02:29:37 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Thu, 28 Jun 2012 17:29:37 -0700 (PDT) Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <4FEB8695.1000009@msapiro.net> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> <4FEB8695.1000009@msapiro.net> Message-ID: <1340929777.19879.YahooMailNeo@web110402.mail.gq1.yahoo.com> How can I make sure that the MTA is properly listening on?SMTPHOST/SMTPPORT (they have the default values, SMTPHOST = 'localhost', ?SMTPPORT = 0)? I installed Postfix by? using the official directions(?http://www.gnu.org/software/mailman/mailman-install/postfix-integration.html?) Also, in regard to the firewall I added a preconfigured rule for the smtp service (port 25), so? I think this should be fine. Furthermore, `netstat -na | grep ":25"` produces the following result:? tcp ? ? ? ?0 ? ? ?0 0.0.0.0:25 ? ? ? ? ? ? ?0.0.0.0:* ? ? ? ? ? ? ? LISTEN? I am just guessing, but could this also be to the `/etc/hosts` folder permissions? This is what `ls -ld /etc/hosts` produces:? -rw-r--r-- 1 root root 281 2012-06-29 03:08 /etc/hosts ? Thanks, Emanuel DANCI ________________________________ From: Mark Sapiro To: Danci Emanuel Cc: "mailman-developers at python.org" Sent: Thursday, June 28, 2012 1:17 AM Subject: Re: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 On 6/27/2012 11:07 AM, Danci Emanuel wrote: > > Also, there is one more thing that could be related to this problem. I also have installed Mailman 2.1.10 > and after setting it up for the proper domain, when a user tries to subscribe to one of the lists, although > the subscription is recorded, the confirmation email is never sent back. The message that appears in > the smtp-failure log is this: > > Jun 04 20:55:57 2012 (10427) Low level smtp error: (111, 'Connection refused'), msgid: > Jun 04 20:55:57 2012 (10427) delivery to esdanci at acm.ro failed with code -1: (111, 'Connection refused') Barry has responded to the MM 3 part of this. With respect to the above, there is no MTA listening on SMTPHOST/SMTPPORT (default licalhost/25) or connection is blocked by a firewall. See the FAQ at . -- Mark Sapiro ? ? ? ? The highway is for gamblers, San Francisco Bay Area, California? ? better use your sense - B. Dylan From mark at msapiro.net Fri Jun 29 02:40:33 2012 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 28 Jun 2012 17:40:33 -0700 Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <1340929777.19879.YahooMailNeo@web110402.mail.gq1.yahoo.com> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> <4FEB8695.1000009@msapiro.net> <1340929777.19879.YahooMailNeo@web110402.mail.gq1.yahoo.com> Message-ID: <4FECF981.9000902@msapiro.net> On 6/28/2012 5:29 PM, Danci Emanuel wrote: > > Furthermore, `netstat -na | grep ":25"` produces the following result: > tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN What happens if you do telnet localhost 25 > I am just guessing, but could this also be to the `/etc/hosts` folder > permissions? This is what > `ls -ld /etc/hosts` produces: > -rw-r--r-- 1 root root 281 2012-06-29 03:08 /etc/hosts So anything can read it, so that shouldn't be an issue. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From danci_emanuel at yahoo.com Fri Jun 29 02:47:53 2012 From: danci_emanuel at yahoo.com (Danci Emanuel) Date: Thu, 28 Jun 2012 17:47:53 -0700 (PDT) Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <4FECF981.9000902@msapiro.net> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> <4FEB8695.1000009@msapiro.net> <1340929777.19879.YahooMailNeo@web110402.mail.gq1.yahoo.com> <4FECF981.9000902@msapiro.net> Message-ID: <1340930873.40897.YahooMailNeo@web110414.mail.gq1.yahoo.com> Exactly what happens with the 9025 one: Trying ::1... telnet: Unable to connect to remote host: Connection refused - executed as root. ? Emanuel? ________________________________ From: Mark Sapiro To: Danci Emanuel Cc: "mailman-developers at python.org" Sent: Friday, June 29, 2012 3:40 AM Subject: Re: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 On 6/28/2012 5:29 PM, Danci Emanuel wrote: > > Furthermore, `netstat -na | grep ":25"` produces the following result: > tcp? ? ? ? 0? ? ? 0 0.0.0.0:25? ? ? ? ? ? ? 0.0.0.0:*? ? ? ? ? ? ? LISTEN What happens if you do telnet localhost 25 > I am just guessing, but could this also be to the `/etc/hosts` folder > permissions? This is what > `ls -ld /etc/hosts` produces: > -rw-r--r-- 1 root root 281 2012-06-29 03:08 /etc/hosts So anything can read it, so that shouldn't be an issue. -- Mark Sapiro ? ? ? ? The highway is for gamblers, San Francisco Bay Area, California? ? better use your sense - B. Dylan From mark at msapiro.net Fri Jun 29 03:05:48 2012 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 28 Jun 2012 18:05:48 -0700 Subject: [Mailman-Developers] smtp error - [Errno 111] Connection refused - when trying to install MM 3.0 In-Reply-To: <1340930873.40897.YahooMailNeo@web110414.mail.gq1.yahoo.com> References: <1340820439.84606.YahooMailNeo@web110410.mail.gq1.yahoo.com> <4FEB8695.1000009@msapiro.net> <1340929777.19879.YahooMailNeo@web110402.mail.gq1.yahoo.com> <4FECF981.9000902@msapiro.net> <1340930873.40897.YahooMailNeo@web110414.mail.gq1.yahoo.com> Message-ID: <4FECFF6C.40006@msapiro.net> On 6/28/2012 5:47 PM, Danci Emanuel wrote: > > Exactly what happens with the 9025 one: > > Trying ::1... > telnet: Unable to connect to remote host: Connection refused > > - executed as root. So it has nothing to do with Mailman. It looks like what's in /etc /hosts is an IPv6 address for localhost You should have something similar to 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan