How to Adopt a Large Framework (was Re: Launching Multiple Servers)

[Peter Hansen]

Andrew Bennetts wrote:
> 
> On Thu, May 22, 2003 at 08:45:29AM -0400, Peter Hansen wrote:
> > Yes, Twisted is large, but there isn't really
> > any trick needed.  Just read the necessary How-Tos ...
>
> We get mixed feedback.  Some people do seem to get overwhelmed by the
> breadth of Twisted, and have trouble finding what they need to know.  Other
> people, like yourself, seem to manage just fine.
> 
> This is mainly conjecture on my part, but as far as I can tell, the main
> reason why some people have trouble is they can't narrow down the docs and
> examples to what matters most to them.  

That's probably true.  And *I* would conjecture that it's because many
people don't really know *what* they want when they start with something
like Twisted.  Actually, the more general rule is that most people don't
really know what they want, period, and it's a problem at the root of the 
sorry state of software development (most projects failing, poor quality, etc.).

I would suggest that one reason why I have no difficulty diving in and
almost immediately getting useful results from something as broad as 
Twisted (or, say, wxPython, or Zope) is that my goals are -- as a direct
result of adopting the XP mentality -- very focused and simple.  

When I started with Twisted, I didn't want to build a massively distributed
multiplayer game with interfaces to MSN and Kerberos or anything like that.
I just wanted to be able to accept a socket connection and disconnect if a 
particular pattern was not received within a timeout.

Only after I got that much working did I start thinking about the next
step, which in this case was to add a check for "too much data received
before authentication", and then "after authentication, connect to this
other thread for further communication".  Short, simple stories, easily
implemented (whether with raw sockets, Medusa, or Twisted).  

As a result of taking baby steps like this, I knew exactly which Howtos
interested me initially (specifically, "Writing a server" under the 
Low-Level Twisted section).  The example code in the Howto looked like 
it should work, so of course the first step would be to verify my 
Twisted installation and the Howto itself by getting the examples to 
execute properly, which it did with no muss or fuss.

At that point, the "Scheduling" Howto seemed particular relevant for
figuring out how to disconnect after a timeout, and Lo, there was the
reactor.callLater() method.

At no point did I worry about the presence of "Cred", or "IM", or 
"Spread", since they didn't seem likely to be of help at that point.
Heck, I still don't really know what they're supposed to do, except
in a vague, hand-waving kind of way.

To anyone struggling with something like Twisted, feeling that it's
"too complex" to get a handle on, I strongly recommend learning more
about agile methods of development, specifically Extreme Programming (XP). 
I'm confident that if you do, you will no longer look on something 
like Twisted with trepidation, but instead with great relief, knowing
that most of the capability you might need will, at the time you need 
it, reveal itself with fairly little effort on your part.

Given the agile manner in which Twisted is developed, maybe it's not
surprising that it can be easily adopted by someone else following an 
agile approach...

-Peter