[Mailman-Developers] PHP Wrappers?

[Brad Knowles]

At 5:18 PM +0900 2005-11-22, Stephen J. Turnbull wrote:

>  To get all the way to "MTA doesn't accept anything that MM is going to
>  refuse to deliver", yes.  But we certainly could define an interface
>  to an auxiliary service such that if the MTA tells us (AuthType,
>  AuthCredentials, EnvelopeSender, Sender, From, To) we'll say "nuke it"
>  or "OK, we'll handle that".  That would allow us to push a lot of
>  simple cases (such as "authenticated user not a list member") back to
>  the MTA, and never invoke the heavy equipment of Mailman itself.

	The problem is that this process is basically what Mailman goes 
through anyway, and I don't see any reason why we would want to 
develop code to duplicate all that functionality -- you'd still have 
to touch all the same Python pickles, deal with file locking 
contention, etc....  Moreover, that's going to be a heavy process, 
and MTAs have some very tight constrictions upon them to get a 
response back to the sender in a reasonable amount of time.


	I agree that the feature is desirable, but short of a complete 
ground-up rewrite, I don't see any way to do it.  Morever, as stated 
in my previous response, I think that a lot of thought needs to go 
into this before we even get to the design stage.

	Given that MM3 is already early in the implementation phase, we 
may have to wait quite a bit longer, especially since the designers 
and developers are few and far between, and what time they have is 
already very tightly spread.

>  Not "have to", but that may be the most effective way to jump start
>  it, may even be necessary for jump starting.  It's not our "job", but
>  it's something we may need to budget resources for if we want to do
>  it.  Of course if those resources are large and won't come from fresh
>  blood, that kills the idea.  But we haven't shown that the needed
>  resources are large, and there are a couple of people who I don't
>  recognize as frequent code contributors who are pushing hard---there
>  might be fresh blood out there to do this.

	If there are people who want to push forward on this now, I think 
they'll have to wait until the specifications and design phase are 
done, before they can start hacking on their favourite MTAs.  Either 
that, or everyone starts inventing their own APIs and hacking their 
MTAs to support those APIs, and then we'll have a half-dozen 
different competing solutions that we've got to try to support.

	People should definitely get some real-world experience in how to 
do this sort of stuff, but no one should have any real hope of 
getting their personal pet version incorporated into Mailman.  The 
Mailman developers might take advantage of their experience in 
designing their own API, but that's about as far as they could hope 
to go.

>  Was it?  I don't know.  My feeling is that Ian is lacking some
>  necessary facts, and consequently expressed his requirements in terms
>  that can't be satisfied.  But if we give him the details and withhold
>  judgment on the substance until he's reformulated, we might find that
>  what he really wants is a lot closer to what would be do-able than our
>  initial impressions are.

	Okay, fair enough.  Let's give him time to re-formulate what he's 
asking for.

>  I think so.  Solving them for Ian probably is not too hard.

	That depends on what MTA Ian is using, and how much code in the 
MTA that Ian is willing (and able) to hack himself, or get someone 
else to hack for him.  Likewise, there will probably be a fair amount 
of hacking on Mailman that would need to be done to support his 
particular requirements, and a lot would depend on who's going to be 
doing that hacking and how much time (and experience) they've got to 
put towards that effort.

>                                                               Solving
>  them in a way that generalizes is going to be hard.  But Python is a
>  very good environment for such generalization.

	The Python part is not the hard part of the equation.  The hard 
parts of the equation are going to be the API specification and 
design process, hacking on the MTA code, etc....

>  Well, you may have a bigger problem in mind, but 90% of Mailman users
>  would get a big bonus from getting only halfway there.  Or maybe you
>  know something I don't.

	I disagree.  I don't think a 50% solution is going to help many 
people, and it certainly won't help them for long.

	It will just push out the event horizon a bit, and since we have 
exponential growth of spam and other forms of mail abuse, it's going 
to be a very short period of time before we're right back where we 
were -- and where there are no more easy 50% solutions possible, and 
where we have yet to spend any time stepping back from the problem a 
bit and working on a better 80-90% solution.

>          Of course you meant something substantially weaker than "all
>  MTAs must understand as an absolute requirement", but surely it was
>  something at least as strong as "we cannot ask users to change MTAs to
>  get this feature."  Agreed?

	If they're using any of the mainstream MTAs, then they should be 
able to expect that they can use virtually all of the features of 
Mailman.  They might be able to use other programs to fulfill other 
components of the system (e.g., the people who use mhonarc instead of 
pipermail), but that should be an optional additional extra 
enhancement that they can add to the base system.

	As I see it, the kind of thing you're talking about would be a 
core component, one that could not be easily changed or substituted, 
and if it's going to be useful at all it will need to support all the 
major MTAs -- or be supported by them.

>  Speaking for myself, I see nothing wrong with asking the bleeding edge
>  adopters to change MTAs, and I see nothing wrong with restricting
>  Mailman-supplied MTA code to a couple of MTAs that we "like".  Of
>  course we help other MTAs to incorporate the feature, but we don't
>  need to promise that they can use it in advance.  Some MTAs may never
>  get it.

	The problem is that there are way too many whackos who then take 
those bleeding edge features and throw them into CPanel or a 
vendor-proprietary version, and then dump off the after-sales support 
onto us.

	And way too many whackos will take whatever code is available and 
blindly install it, without really understanding that they need to 
have a certain minimum level of competence before they take bleeding 
edge code and try to start working with it.


	Many vendors restrict who has access to development beta code, 
and for good reason.  If you're a MM3 developer or want to be, then 
you should be free to play with that code about as much as you want 
-- after all, the only risk should be to you and your mailing lists.

	But you should anticipate that many people may expect you to hack 
on the code yourself to fix whatever problems you may end up creating 
for yourself.

>  OTOH, I wonder if you're not overestimating the degree of difficulty
>  and underestimating the benefits of the structural work required.

	I don't think I'm underestimating the benefits of the structural 
work.  We won't know whether I'm overestimating the degree of 
difficulty until the actual work is done and we can look back and see 
what it took to get where we did.

>  The "database backend" is _not_ structured that way yet,

	Which is a large component of my caution on this topic.  Code 
hacking is easy.  Database hacking is harder, because you've not only 
got to hack the code which accesses the database, but also the 
database structures that would be accessed.  More importantly, since 
you're talking about exposing some aspect of this database structure 
to the outside world, you can't just take the easy way out and 
directly access everything but you instead have to work through a 
published API layer.

	In programming, global variables make a lot of things easier in 
one aspect, but they greatly complicate life in others.  It takes 
more work to write code in such a way to avoid global variables, but 
in the end it usually pays off.

	Working through published API layers has a lot of benefits too, 
but it also takes a lot more work to make that happen -- compare the 
classic C version of "Hello, world" to what you have to do to get the 
same results with a standard X11 library.  The source of the X11 
version is likely to be many times larger than the compiled code of 
the C version.


	In this case, you're asking us to create the equivalent of the 
X11 API and library, plus the associated database structures, etc....

	Yes, we most likely have a smaller overall problem space, but we 
also have many fewer programmers, each of whom is probably spending 
less time on this work, and there are many classes of pre-existing 
programs that we will have to somehow find interoperability with.

-- 
Brad Knowles, <brad at stop.mail-abuse.org>

"Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety."

     -- Benjamin Franklin (1706-1790), reply of the Pennsylvania
     Assembly to the Governor, November 11, 1755

   SAGE member since 1995.  See <http://www.sage.org/> for more info.