Even before my first experience with full-on eXtreme Programming (XP), I was mulling the problem of how much design to do when. I’ve seen too many waterfallish projects either throw away much of their early design work, or go chaotic late in the schedule trying to cling to an early design in the face of a new reality that needs something different. When defending waterfall, it’s almost cliche to trot out some variation on Eisenhower’s famous saying, “In preparing for battle I have always found that plans are useless, but planning is indispensable.” I used to nod sagely, but now I see it as a poor excuse for the amount of waste that happens when you try to do the bulk of your design work up front.
XP seems to cure waste by spreading out design over the course of the project, doing only enough to satisfy the work at hand, followed by aggressive refactoring. Working on an XP project, I found, to my pleasant surprise, that this “small steps” approach worked very, very well. Doing small bursts of design, writing tests first, coding up the simplest implementation that would satisfy those tests, and then stopping to clean house by refactoring, proved to be an highly effective way to develop. Except when it didn’t.
Working a story (use case) at a time keeps us focused. Having a large body of tests gives us confidence that we aren’t screwing something else up (and nudges us towards writing testable code). Pairing helps spread domain and technical knowledge around. Refactoring keeps the code base clean. But this isn’t always enough to prevent design debt from building up.
There are points in the evolution of some systems where something happens that changes the game. It may be the introduction of a new requirement, such as “now make it work on the web”, “support simultaneous edits”, or “accept foreign currency”. Or it may happen when increased demand forces our system to expand beyond a single server. However it happens, the game has changed in some fundamental way, and the code base we have, or some significant part of it, isn’t structured in way that fully supports the direction we now find ourselves headed. We could try continuing to move ahead in small steps, but some of those steps get harder and harder to take. It’s time to invest a larger chunk of time in design, with the intent of getting ourselves back on the “small incremental design” track.
So plan to spread out your design, but don’t expect to spread it smoothly like peanut butter. Expect some big chunks.
On a slightly related note, this video interview with Ron Jeffries is excellent. It’s mostly about dealing with fixed dates and the importance of testing, but he makes some points about spreading out design, and not expecting it to spread evenly, that might be helpful if you find yourself in setting expectations about using Agile.