Why do Pythoneers reinvent the wheel?

[Dave Brueck]

Stefano Masini wrote:
> I wonder how many people (including myself) have implemented their own
> versions of such modules, at least once in their pythonic life. I
> indeed have my own odict (even same name! :). My own pathutils
> (different name, but same stuff). My own validate... and so forth.
> 
> This is just too bad.
> There are a few ares where everybody seems to be implementing their
> own stuff over and over: logging, file handling, ordered dictionaries,
> data serialization, and maybe a few more.
> I don't know what's the ultimate problem, but I think there are 3 main reasons:
> 1) poor communication inside the community (mhm... arguable)
> 2) lack of a rich standard library (I heard this more than once)
> 3) python is such an easy language that the "I'll do it myself" evil
> side lying hidden inside each one of us comes up a little too often,
> and prevents from spending more time on research of what's available.

IMO the reason is something similar to #3 (above and beyond #1 and #2 by a long 
shot). The cost of developing _exactly_ what you need often is (or at least 
*appears* to be) the same as or lower than bending to use what somebody else has 
already built.

(my wheel reinvention has typically covered config files, logging, and simple 
HTTP request/response/header processing)

> It seems to me that this tendency is hurting python

I think it helps on the side of innovation - the cost of exploring new ideas is 
cheaper than in many other languages, so in theory the community should be able 
to stumble upon truly great ways of doing things faster than would otherwise be 
possible. The problem lies in knowing when we've found that really good way of 
doing something, and then nudging more and more people to use it and refine it 
without turning it into a bloated one-size-fits-all solution.

I think we have half of what we need - people like Fuzzyman coming up with handy 
modules and then making them available for others to use. But right now it's 
hard for a developer to wade through all the available choices out there and 
know which one to pick.

Maybe instead of being included in the standard library, some modules could at 
least attain some "recommended" status by the community. You can't exactly tell 
people to stop working on their pet project because it's not special or 
different enough from some other solution, so maybe the solution is to go the 
other direction and single out some of the really best ones, and hope that the 
really good projects can begin to gain more momentum.

For example, there are several choices available to you if you need to create a 
standalone Windows executable; if it were up to me I'd label py2exe "blessed by 
the BDFL!", ask the other tool builders to justify the existence of their 
alternatives, and then ask them to consider joining forces and working on py2exe 
instead. But of course I'm _not_ in charge, I don't even know if the BDFL likes 
py2exe, and it can be really tough knowing which 1 or 2 solutions should receive 
recommended status.

FWIW, RubyOnRails vs all the Python web frameworks is exactly what you're 
talking about. What makes ROR great has little to do with technology as far as I 
can tell, it's all about lots of people pooling their efforts - some of them 
probably not seeing things develop precisely as they'd prefer, but remaining 
willing to contribute anyway.

Many projects (Python-related or not) often seem to lack precisely what has 
helped Python itself evolve so well - a single person with decision power who is 
also trusted enough to make good decisions, such that when disagreements arise 
they don't typically end in the project being forked (the number of times people 
disagreed but continued to contribute to Python is far higher than the number of 
times they left to form Prothon, Ruby, and so on).

In the end, domain-specific BDFLs and their projects just might have to buble to 
the top on their own, so maybe the best thing to do is find the project you 
think is the best and then begin contributing and promoting it.

> and I wonder if
> there is something that could be done about it. I once followed a
> discussion about placing one of the available third party modules for
> file handling inside the standard library. I can't remember its name
> right now, but the discussion quickly became hot with considerations
> about the module not being "right" enough to fit the standard library.

I think an extremely rich standard library is both a blessing and a curse. It's 
so handy to have what you need already there, but as you point out it becomes 
quite a debate to know what should be added. For one, a module to be added needs 
to be sufficiently broad in scope and power to be widely useful, but this often 
breeds complexity (e.g. the logging package added in Py2.3 sure looks powerful, 
but other than playing around with it for a few minutes I've never used it in a 
real app because it's a little overwhelming and it seems easier to just use a 
quickie logging function that does all I need).

Having two versions of the standard lib probably wouldn't solve anything - you'd 
still have debates about what goes in the "lite" version, but you'd also have 
debates about what to include in the big version - maybe even moreso.

-Dave