[Distutils] Python people want CPAN and how the latter came about

[Steffen Mueller]

Hi Lennart,

Lennart Regebro <regebro <at> gmail.com> writes:
> On Wed, Dec 23, 2009 at 10:32, S. Mueller wrote:
> >> I have to say that I vastly prefer not to have any authorization and
> >> allow anyone to release anything in any namespace. But then I am
> >> getting fanatically anarchical in these issues. You can not organize
> >> freedom.
> >
> > But you have to.

This is clearly a case of citation rape. ;)

> No, I really mean it whan I say you can't. And you never *have* to do
> the impossible, and trying just leads to problems. I realize this is a
> matter of attitude, but if the sentence "I want CPAN" means "I want
> restrictions and controls and people preventing others from uploading
> stuff", then they are misguided.

Sorry, but I'm not being philosophical when I say you have to authorize access
to things. Apparently the Python repository does, too. Or otherwise I'll upload
a few popular packages with high version numbers that contain viruses for New
Year. I'm not talking about restricting freedom.

Now, on CPAN, I *can* upload anything even if not authorized to do so. It just
won't be part of the official indexes if I upload a new version of the database
interface DBI.

> > What you're saying here means you virtually throw
> > away the ability to do anything useful with namespace meta data.
> 
> *shrugs* Namespaces has no metadata in Python. They are just
> namespaces, no more, no less. The names of your *packages* is
> protected on PyPI. But several people can use the same *namespace*.
> Ie, nobody can upload a "Twisted" package, except those who have the
> permission. But people can upload a "zope.my.own.package", even though
> the namespace "zope" is already used by other packages.

Same for CPAN. You automatically register an exact namespace by uploading a file
that contains it. But you don't get it recursively. Please recall my explanation
of how in Perl a namespace == package name == class name. If I upload
Steffens::Module, somebody else can upload Steffens::Module::Plugin. Just
Steffens::Module is restricted from the point on of the initial upload.

That we do out meta data stuff on package/namespace/class names as opposed to
distribution names has the huge benefit of interoperability between
distributions.

> > Think about it like this: If you install any module from CPAN (and
> > only the valid ones end up in the index), you can use all of them in
> > the same application. If module A and B could both implement
> > Config::Parser, then you couldn't use both of them at the same time.
> 
> This would be true for Python too. But Python doesn't try to pretend
> that all the packages that exist are some sort of standard library,
> and therefore don't try to put them all into one sort of hierarchical
> organized namespace. And to be honest I don't see the point of doing
> that.

We're not pretending anything. We're not forcing anything except that you don't
override somebody elses work. We advise on proper choice of namespaces. But in
the end, we never force anybody to adhere to our preferences. By "our", I mean
an arbitrary bunch of experienced contributors who offer advice for new
contributors. Most of those wouldn't even have the power to impose any
restrictions. Those who do, use it only extremely sparely. For example, when
somebody has passed away or simply asks for help in passing maintainership to
someone else.

> > Still. We allow for a lot of creative freedom. We just don't want a
> > random newbie upload a shiny package "DB" which implements his idea of
> > a database interface when it's really the name of the package that
> > implements the Perl debugger. He can still upload. The file will be
> > accessible in his CPAN directory. Users can install it from the CPAN
> > shell as "install NEWBIEUSERID/DB-1.00.tar.gz", but not as "install
> > DB" or "$ cpan DB".
> 
> I see. But IMO Perl then there starts out with trying to organize
> freedom from the start, and that then leads to the above problem that
> newbies can come up and mess up this so called organized freedom,
> which means you need to restrict it even more by having people control
> and restrict the namespaces, etc, etc. You end up having to have more
> organisation to fix the problems your organisation caused. This is,
> without trying to be rude or anything", the fate of all bureaucracies.

You're wrong. The Perl/PAUSE/CPAN system works exceptionally well. But it does
so because we regulate a lot less than you think. Let me recap the major points
once more:

a) Anybody can upload anything (theoretically, even you wedding pictures)
a1) If it's a virus and identified, it'll be deleted.
a2) Anything else goes into CPAN.
b) PAUSE extracts archives and checks whether it looks like a Perl module
distribution.
b1) If not, it's simply ignored (but still mirrored)
b2) If so, it's scanned for Perl classes. (== namespaces == packages)
c) The classes found are added to the official index.
c1) This is the case if the classes have never been uploaded before.
c2) Also, if *you* uploaded them before.
c3) Or if the guy who first did transfered maintenance or assigned you as
co-maintainer.
c4) If somebody else offers a class named $foo, your distribution enters CPAN,
but doesn't get included in the indexes until the clash has been resolved.

Now, c4) takes care of newbies "messing up the organized freedom". It's a
safeguard for users who expect DBI.pm to always be Tim Bunce's DBI.pm. But it
might as well be a bit of legal protection, too.

> > This is a safeguard against insanity and it's the thing that means
> > that you can trust "cpan PAR" to always install the Perl Archive
> > Toolkit that was released by Autrijus Tang, Roderich Schupp, or myself
> > (we share co-maintenance). It's never going to be some random junk.
> > And that you auto-register a namespace on upload is the guarantee.
> 
> And obviously on PyPI, it's first come first serve as well. But nobody
> would call a db package "db" if one already exists. Why would they do
> that? What's the point? Why would I make a completely new package
> called "Twisted" for example? There already is one. It's just a
> mindset that is completely incomprehensible to me.

Then you clearly do not understand what it is like to be
a) malicious
b) new, young, inexperienced
c) stupid.

> I expect what I would call creative freedom, you would call total anarchy.

No. But you did yourself in an earlier mail. :)

> > PS: Let me comment on another post of yours quickly. No. In the Perl
> > community, the name "CPAN" doesn't yield confusion. It's just a way to
> > refer to the whole ecosystem
> 
> OK, that's not how it sounded in your first post, thanks for clarifying.

Sorry if I caused confusion.

Best regards,
Steffen