Jeremy Hylton : weblog : June 2003 last modified Thu Mar 17 01:11:16 2005

Jeremy Hylton's Web Log, June 2003

XP and Design

permanent link
Thursday, June 05, 2003

Are "bad smells" in extreme programming projects caused by a lack of up-front design? Elssamadisy and Schalliol from ThoughtWorks wrote a paper describing problems they had in a large (50-person) project that used XP.

One example from the paper is a project where a large refactoring project was delayed on grounds of doing the simplest thing. They believed that they would eventually need general framework, but at each iteration they made only minimal steps towards it.

Recognizing Bad Smells in XP:

What we SHOULD have done is realize that it is OK to look ahead and trust our inner-instinct that we are going down the wrong path. We should have continued to do thing incrementally, but after the first instance we should have pulled the work and made it into a factory instead.

The authors later argue that

if you find yourself having to do a large refactoring then you should have done many smaller refactorings earlier and you got lazy.

It seems just as likely that you should have spent more time designing the framework upfront. I think you ain't gonna need it (YAGNI) is a fine principle, but I believe there are also cases where you know you will need it. If you know you will need it, then taking many small incremental steps is probably more expensive that spending a little more time upfront to get the design right.

I saw a reference to the paper in the June issue of IEEE Computer has a collection of articles on "agile software development." There's a bizarre debate between Kent Beck and Barry Boehm. There is also an interesting history of iterative and incremental development. It sounds like the less extreme parts of XP have been around for a long time. When I first heard about XP, it reminded me of Brooks's advice:

Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time.


permanent link
Wednesday, June 18, 2003

Raymond Hettinger's new iterools module has a wonderful collection of features. Iterators and generators seem like they will make some big changes in Python style.

I haven't had much excuse to play with itertools, but a post by Guy Steele on the lightweight languages list gave me an excuse to try it. Steele wrote:

Sometimes an circular structure can be used to "advantage". Consider this bit of code:

(mapcar #'+ x '#1=(1 -1 . #1#))

If x is (1 4 7 10), the result will be (2 3 8 9).

Does this puzzle you? You get 1 point. Does this delight you? You get 5 points. Does this nauseate you? You get 10 points.

The Python version is a bit more verbose. Python doesn't have the nifty syntax for creating recursive objects. In this specific case, the itertools cycle() function suffices.

from itertools import imap, cycle
tuple(imap(int.__add__, (1, 4, 7, 10), cycle((1, -1)))

imap() is an improvement over the builtin map. It doesn't run until it has consumed the longest sequence, which wouldn't make sense for the infinite sequence returned by cycle().

We get to use int.__add__. This is one of my favorite tricks from the type-class unification. There's no need for the operator.add if all the objects are of the same type. Just pass the type's __add__ method.

There is a bit of extra typing required to see the output. An iterator doesn't have a useful repr() for interactive use, so you need to pass it to list() or tuple() or a for loop. Of course, I wouldn't want cycle() to have a "useful repr().

"Your blog software sucks"

permanent link
Friday, June 20, 2003

Barry said that today, and I have to agree. I settled on BlogMax because I couldn't find much blog software that just produced static pages. It seemed valuable to have the software integrated with XEmacs, since that's what I use to write it anyway.

I had to write my own RSS feed generation, because the one with Blogmax doesn't even come close to working. What I'm left with is the template system and the emacs key bindings to generate HTML from my simple text input files. Even the input format is a bit lame: The title needs to be put in an HTML comment, and any inline markup needs to be done in HTML. I'd rather use reST or plain text.

The software has a one-entry, one-day model. It seems to be hard to add multiple entries on the same day, but with different timestamps. One consequence of that software limitation is that I don't write short entries. If I'd write a short entry, I'd use up the whole day's worth of material.

Zope3 Tutorial

permanent link
Saturday, June 21, 2003

I took the Zope3 tutorial by myself today. It is coming along nicely, but still not ready for the home schooler. There are a bunch of bugs in the code in CVS and in the slides that make it difficult to make progress without an expert at hand.

I've taken the tutorial before, but probably not in about a year. It was a helpful refresher.

Programmer Tutorial

The code in CVS doesn't match the code in the slides. One example so far: The contact configure.zcml in CVS specifies a permission, but the slides don't. As a result, I burned an hour trying to figure out what was going wrong. (In part, my own fault for doing the tutorial on my own instead of having Jim teach it.)

I ended up having the same problem with the manage content permission, which lead to a separate problem -- component lookup error. The type showed up in the add menu, but I got the error after I chose a name for it. It wasn't obvious why a component lookup error would occur adding something.

There seem to be some capitalization problems. The README.txt, slides, and code differ about what to capitalize. It shouldn't matter, but since it does, it ought to be convenient.

The slides tend to move a bit quickly. I'm sure Jim explains more and demonstrates what happens when he gives the tutorial.

Step2 says it is about security assertions, but it seems to be more about views that an about security. Or maybe it isn't about views. It shows the, but doesn't seem to wire it up. What is it for?

In Step3, we create a new, so I'm not sure what the point was. I don't know what went wrong here, but I can't get the contact to show up in the add menu. It doesn't require a permission; I checked by adding a print in the global menu service thingy. I can add it manually by passing the type_name to @@contents.html. Good enough for 11pm.

Note: It looks like the browser menu action attribute is a name in the namespace of factory ids. In most menus, the action is a relative URL. The adding code knows to interpret it as a factory name if it isn't a view name.

While Step3 didn't work, Step4 did. Go figure.


The configuration code is simpler than the last time I looked at it. I wanted to verify that my contact config was getting loaded. The Contact object wasn't showing up in the add menu, but I wasn't sure why.

The XMLConfig class provides the basic machinery. Unfortunately, it doesn't have any comments or docstrings. The _include() method is called for each file, so a print there confirms that the config is getting loaded. (Ought to have a debug log call there.)

The actual files are parsed using SAX. The ConfigurationHandler class handles the data as it is processed. It accumulates configuration directives into __actions, which is a container passed in by the caller. The caller gets the results by virtue of side-effects on the container. (Not the clearest way to right that code; definitely deserves a comment.)

The handler looks up directives in a registry to find the method/function that should be called. The caller has the loop that actually defines the call.

It's still hard to find the code that handles the individual directives. XXX

User interface

>From the @@contents.html page, click on a content type to add. A new page is loaded with a text box to enter the name of the type, but there is no indication of what type. If you clicked the wrong thing, you'd never know. One solution would be to show the same menu with the selection highlighted.

The security grants page doesn't provide a way to get back to the rest of the management UI.


All lot of my early trouble had to do with security problems. When they occur, they don't seem to leave any trace. There's no logging or error message to indicate that something went wrong because of a permissions problem. In a production site, that might be the right thing. (By default, don't reveal too much information by announcing missing permissions.) But it makes it very difficult to developer or debug.

It might be interesting to have a special debug mode where you get to see all the permissions that were used to allow something to happen.


Slide 14: "Specify the interface of the container adding component." What does that mean?

When you're making something internationalize-able, you add an attribute i18n:translate=""? Are the quotes just for syntactic completeness? That is, is the mere presence of the attribute sufficient to mark it.

Sprint notes, day 1

permanent link
Sunday, June 22, 2003

There was a small sprint at Zope Corp. world headquarters today -- Jim Fulton, Chris McDonough, Paul Winkler, and me. I took a few notes as we were getting started.

Jim observed that we haven't focused on the non-programmer user of Zope. These are the people who made Zope2 successful. Zope2 didn't succeed with Python programmers. Zope3 focuses on Python programmers, but we need to make sure it is useful for non-programmers.

One important thing about the non-programmer user is that they have a problem they are trying to solve. They may understand a problem domain much better than a programmer who must gather requirements first. Zope2 allowed these novice programmers can build very important software without understanding a lot about software development. They know what they need to accomplish, if not the best software practices for accomplishing it.

Chris observed that its possible to create an application using relatively little framework. The AARP application uses basic CMF but the rest is mostly Python scripts. A framework may be useful for a user who understands it deeply, but a lot of users want to build their application without investing a lot of time learning the framework.

In my tutorial notes, I mentioned that security failures are particularly inscrutable. Apparently, there is a checker logging service that Shane implemented. Jim thinks its controlled by an environment variable.

What is registration? Local software: Stored in ZODB, but developed as much like regular Python software as possible. Distributed via bundles -- contains the software and the configuration. Registration is "the configuration of how an object will be used." Many services are registries. Adapter service registers factories by interface. Example: postal lookup utilities. You can have multiple registrations for the utility, one active, the others inactive. If you install a new utility and find it has a problem, you should be able to uninstall it and have the old utility become the active registration. Services that require registration are a lot harder to write.


permanent link
Monday, June 23, 2003

Michael Yoon posted an interesting reflection on the demise of ArsDigita.

It is a more balanced account than Eve Andersson's Diary of a Start-Up or Philip Greenspun's >From Start-Up to Bust-Up. Yoon explains bad decisions that were made, including some of his own. The other pieces tend to assign blame without explaining the nuts-and-bolts of what was happening.

There are some parallels with Zope Corporation. I think ZC has avoided a number of the problems that hurt ArsDigita. Many or most of AD's customers were themselves startup companies; ZC has a more diverse customer list. ZC has stayed small, instead of ballooning up to 100s of employees in anticipation of future revenue. This difference may be a matter of timing. If ZC had gotten a big investment at about the time AD did, ZC might have grown too big, too.

Zope Corp. has done a much better job of engaging with the larger open source community. AD seems to have been at odds with the developers working on OpenACS. The core Zope software is managed by ZC, but there are many external contributors.

The story of ACS4 should be a cautionary tale for Zope3, although I think it's possible to manage the Zope3 transition better.