Ripping out old code is satisfying, but it’s a lot harder to get it out than it was to put it there in the first place, especially in a well-factored system with lots of small methods. Ripping out the code close to the “surface” (i.e., API or UI) is easy. Making sure that you’ve tracked down all of the newly-orphaned methods in deeper layers can involve a lot of tedious work.
I have a feeling that there’s a way to leverage TDD in reverse, but I’m not seeing it just yet.