Code Reviews: Over-rated

According to Rapid Development, studies have shown that code reviews are beneficial: in theory, the number of bugs caught early in the development process justify the amount of work put into reviewing the code. So we tried it. We wanted to do it right: I read two books on reviews, one by Weinberg and one by Glib. We reviewed, counted the caught bugs, estimated how much time we saved by finding the bugs early.

Our finding? Although we did find some "bugs" early, most of them were not stop shipment bugs. Readability bugs, guideline violations, sure. Every now and then, maybe every other review, we found an actual stop shipment bug and were like, Yes! Good catch!

But we estimated that the amount of time we would have spent fixing the stop-shipment bugs later was actually less than the amount of time we spent reviewing code.

So why are code reviews so damn popular in these studies and software engineering books? My theory is that, way back when, when our compilers and languages weren't so good at catching bugs at compile time, and common practices such as Steve Maguire's for catching bugs at runtime weren't so common, code reviews were extremely beneficial. But as time went on, they became less and less so, until finally the cost outweighed the benefit.