A frequent argument between people who practice Test-Driven Development (TDD) and those who don’t is whether the overhead of all those extra tests slows a project down or speeds it up. Now there’s a study to cite.
Exploding Software Myths, an article from Microsoft Research, summarizes some recent research in software development, including a study, Realizing quality improvement through test driven development: results and experiences of four industrial teams, which compares development done using TDD to “normal” projects, using data gathered from projects at Microsoft and IBM. The upshot?
“What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer.”
So there it is. TDD is slower.
Now consider an alternate story, told using the same numbers:
By doing normal development instead of TDD, teams can complete projects 14 to 26 percent faster at the expense of defect densities that are 150 to 900 percent higher.
In a short lightning interview that Ward Cunningham did with with Ron Jeffries and Chet Hendrickson, Chet has this to say about Technical Debt (emphasis mine):
“Our experience has been that by the time you start to notice that the code needs to be cleaned up, you’ve already lost a lot of time because of it, and it’s almost impossible to ever get back to where you were. We actually believe that you can never be as clean by going back a month or a week later and cleaning up than you could if you just cleaned up as you went.”
“By the time you start to notice” is a challenge. You either have to be so clean in your coding practices that you never present yourself with the need to notice problems, or you have to develop your ability to notice problems so that you can spot them sooner. I’ve never seen a team—even one doing TDD and refactoring in earnest—manage to stay completely clean, so it would seem that developing noticing skills, and helping teams develop willpower, is the way to go.
On the chance that you’re a West Coast Agilist and missed the memo, Agile Open California is coming up next week on Monday and Tuesday, October 6 & 7, in San Bruno.
I had to miss Agile Open California 2007, and heard great things about it. I’m looking forward to it this year.
When coaching teams, one of the problems that I often run into is how to move the team’s definition of “done” past “it works!” to a place that includes those little things like code and test cleanup. A lot of technical debt can pile up in the gap between those two places.
To motivate moving the definition, I’ve used the analogy “Leaving a Clean Campsite”. When I was a Scout, the standard procedure on breaking camp was to form a line abreast and walk through the campsite removing all traces of humans having been there, even if they were traces that we hadn’t left. If someone else left a gum wrapper, we’d clean it up. The goal was to leave the campsite a cleaner place than we’d found for the benefit of the next set of campers.
I believe in the same approach for code: Before checking in a change, make a pass through the affected files looking for refactorings and other cleanups. A bit of new code isn’t as simple as it can be? Refactor it. Someone else left a formatting problem or a typo in a comment? Clean it up. Try to leave the code a better place for your having been there. I’ve found that this incremental attention to cleanliness and detail leads to a more habitable, enjoyable code base.
The problem with the “Clean Campsite” analogy is that it doesn’t translate well to a time and place where camping isn’t part of the common shared experience. I was getting more blank looks. The message wasn’t getting across.
So earlier this week when I gave a short presentation to a team that I’d been was working with, I tried two analogies: “Leave a Clean Campsite” and “Leave a Clean Kitchen”.
The Clean Kitchen won. People who sat quietly through the campsite nodded in agreement at the kitchen. When the meeting notes came out, “clean kitchen” was what got recorded. There was no mention of some old campsite.
No surprise, really. More people have been in a kitchen than have been camping. Even if they don’t cook, they get that a clean kitchen is an easier place to start working in than a dirty one. And if they have cooked, they really get it.
So that’s is the analogy that I’ll stick with for now.
I’ve been happy in the past with shirts that I’ve bought from REI. Their shirts are usually comfortable and sturdy, and look funky in a way that is still safe for corporate casual. But fashion is all about change, and sometimes change has unexpected collateral damage.
After a season of zippered and velcroed shirt pockets, the cool thing in outdoor-wear this season is to either omit the pocket altogether (huh?), or to stitch a separator line vertically through the chest pocket to form two separate partitions. I guess this is to make it harder for sunglasses to fall out, which could be a feature when dangling from a cliff or trying to outrun a bear. Unfortunately, for the urban programmer type this means a shirt pocket that is too narrow to hold that most essential piece of Agile equipment, the 3×5 index card.
I’ve seen teams “game” velocity, doing risky things (like letting unit testing slip and quality drop) to keep their velocity high. It didn’t occur to me that part of the problem might be terminology. Tim Ottinger’s “Velocity is Just Capacity” (stale link, but here’s Tim’s later thinking) gave my thinking a serious nudge.
“Velocity” evokes speed and quickness. Speed and quickness are good. Speed and quickness win races and awards. A manager whose team has a high velocity is a winner. Recall Mike Cohn’s advice to alway append “billion” onto any velocity number that you give to a manager so that they can one-up their peers.
But consider the reframe: A team has a demonstrated “capacity” for taking on work. The prior iteration (or a weighted average of the past few iterations), establishes a Load line (AKA “Plimsoll line”) around the team. Load is a measure of capacity. It’s velocity as Agile uses the term, but without the baggage of evoking competition. Discussions about increasing capacity are subtley different from discussion about increasing speed. Tim lays it out nicely in his article. I think he’s on to something.
Is a terminology shift all we need to drive out bad velocity behavior? Sadly, no, but I think it would be step in a good direction.