Jamie Test, Step 8

Let team members estimate their own schedules

I really don't have much to say about this that Joel Spolsky and Erik Bethke and Steve McConnell didn't already cover.

I have a friend who has a friend who is working on a high-profile game who is afraid that they don't have enough programmer bandwidth to complete their feature set. I say to that friend: "A Hobbit Can Contend With The Will of Sauron." If you are at a company where they do not ask you to estimate your own schedule -- and in my opinion even asking something like "Do you think you can get this done by Friday?" counts as not asking, because programmers are notoriously underconfident in their estimating abilities and would rather pass the responsibility upstream -- then find some time, either on a weekend or when the boss is not looking, to write down a list of everything you have to do before you ship, and estimate how long each thing is going to take. Then add the various buffers Joel suggests in his article. (25% debugging, 10% unplanned features, etc.) Then compare your results to your project's "alpha" or "feature complete" date. You will probably find you're Not Going To Make It. Take those findings to your lead and ask to have features transferred from you to the other programmers. Recommend that you have the other programmers do the same thing, to make sure that they're going to make it. You will probably find that none of you are Going To Make It. At that point, reccomend that they ask their publisher for a) More Money, b) More Time, c) Less Features, or d) some combination of the above.

Your lead won't want to hear it. He may say something like, "How do *you* know this is how long it will take?" He may say something like, "I think you're just lazy!" Remember your people skills. Say things like "I'm concerned about the accuracy of our schedule" rather than "This is fucking bullshit!" (I'm not very good about this, myself. But I try to save the bullshit-calls for emergencies.) How you handle this conversation could mean the difference between saving the project and ruining it. If he's a Good Lead, he won't kill the messenger, and will take your advice, and possibly save the project. (Peter Drucker says good leaders are secure enough in their leadership that they foster disagreement among their 'followers'.) If he's a Bad Lead...then I don't know what to do next. Maybe you can go over his head, or maybe your team has a way you can give feedback anonymously.

I've never been in this exact situation myself. I have been in the situation where I was handed a project--I was the only programmer--and the first thing I did was come up with estimates and it turned out we didn't have enough time. I immediately told the producer, who happened to be a good friend of mine who got me the job in the first place. He asked me if I'd be willing to put in overtime and I said no. They accepted that and threw more money at the project. Later, when an unplanned feature cropped up, and the code I'd have to be dealing with was cryptic stuff from another dimension, I flatly refused to add the feature. The boss himself came by and asked me if I'd reconsider. Here I was more stubborn than I should have been: I didn't even give them an estimate. I said it was impossible. Honestly, given some amount of time, I could have solved the problem, and once I told them that amount, they probably would have agreed that the bang wasn't worth the buck. On another project with the same producer, we were in the last few weeks and I realized we weren't going to make it. They found another programmer and together we managed to break Brooks's Law and finish on time. And there have definitely been times were it seemed like upper management was trying to push my team around and I pushed back. (Probably could have handled those times better.)

The thing is, times were different then. Game programmers were in short supply or high demand, I'm not sure which. I didn't have to worry about finding another job. If I had to (and it never did come to that) I could have voted with my feet. These days, with the major publishers' ubiquitous mantra of "fewer better games" it may be tough to land that next job. So don't be as reckless as I would have been. Make a sincere effort to institute change, don't flip out, and, if your effort fails, send out your resume. (But be careful about it. There's only a few degrees of separation in our industry. There's a good chance that the person you send your resume to is friends with someone on your team! And your headhunter does not care! Very few people at Treyarch send out their resume without me finding out about it.)

A hobbit can contend with the will of Sauron.

Jamie Test, Step 7

Fix Bugs Before Writing New Code

This is the central theme of Steve Maguire's *Debugging the Development Process*, the first book on project management that I ever read. You never forget your first book on project management. It is also straight off The Joel Test. Go there if you want to find out why you should fix bugs first.

I'm here to tell you, in the games industry, most people don't seem to know this. There's a misconception that alpha is the time we fix bugs; up until then we write features, allow a few bugs to get in, and when we hit alpha we fix them all. You'll hear various specious logic about why to skimp on the bug-fixing, such as, "If I take time to fix the bugs, I won't get this feature done on time," and "There's always more bugs! If we tried to fix them all, we'd never be able to write any features!" This is complicated by upper management wanting to see visible progress all the time; they notice when you get the lens flare in, but they don't notice when those fall-out-of-the-world bugs stop happening.

But blaming upper management is too easy. The real problem is us; we hate finding and fixing bugs. We'd rather stick needles in our eyes. It drains the life out of us. It's Sisyphean; every time we fix a bug, two new ones seem to show up on our lists. So we're all too happy to agree when a manager says, "We need to get the minke in there by Friday," because that means we can stop trying to find whatever damn bug we're supposed to be finding, and actually write some new code.

The problem is further complicated by the general pattern which we do things: we code a feature, we check it in, we start on a new feature. While working on the new feature, bugs get reported about our last feature. So we've already broken the "Don't write new code before fixing bugs" rule.

So: how do we do it? How do we change our team so that we fix bugs first?

For what it's worth, here's my advice. Take this advice with a grain of salt, because my team is far from exemplary when it comes to getting those bugs fixed. With our last project, we found at least six thousand bugs (eighteen thousand if you count duplicates) during the alpha phase. That's six thousand bugs that probably should have been found and fixed sooner.)

1) Be flexible in your scheduling. Joel's scheduling system has flexibility built into it. Milestone schedules do not. If you do milestones, then an important criteria for a succesful milestone should be no bugs. It's well-nigh impossible to do zero defect milestones on a monthly basis, IMO; Microsoft's Visual C++ team did them quarterly. Schedule at least 25% of your time for bug fixing. Slip features on milestones rather than slipping bugs. If management pushes you around for not meeting targets, push back.

2) Track bugs from the beginning of the project. Don't wait for it to go into testing. This seems brain-dead obvious to me but a lot of teams don't do it.

3) Have testers from the beginning of the project. If you don't have testers, do the testing yourself. Set up a system where anybody on the team can submit a bug. Schedule time to do testing.

4) You still need to perform triage on your bugs. (http://www.joelonsoftware.com/articles/fog0000000014.html) If you fix every problem everybody has with every feature, it is true; that feature will never be finished. Somebody has to ask themselves: "Could we ship this bug in the box if we had to?" If the answer is yes, the bug goes on the low-priority list, where it doesn't get fixed until alpha. Which means a lot of cosmetic bugs - or graphical glitches - or game design flaws - are high priority bugs that must be fixed when they come up, unless you're willing to ship them.

5) Make sure everyone on the team understands that if they have bugs on their list, they are supposed to stop what they're working on and work on those bugs. This will not make them happy. The knowledge that they're increasing the likelihood of shipping on time is little consolation.

6) Make your asserts fatal. The team will come to a grinding halt when a bug is introduced. Believe it or not, this is a good thing, because people fix their bugs in a hurry when others are waiting on them.

One thing we've tried that Does Not Work is code reviews or inspections. "What?" you say. "But in *Rapid Development*, Steve McConnell said they were the cooler than unit tests! You must have done them wrong!" Honestly; we did it by the book. We read both Glib and Weinberg and instituted a formal policy. We counted the number of stop-shipment bugs we found. By our estimates, the time we spent doing the inspections was greater than the time we would have saved fixing those bugs later, when they popped up. (Although we were counting just coder time when making the estimation. The cost of bugs to producers and testing may have made inspections worth it.) It is my feeling that inspections were something that was more useful when languages and compilers were more prone to letting bugs get through; now that we have automated warning messages and higher level constructs and asserts and the like, inspections don't catch that much. Inspections are also good for enforcing coding standards, if you really care about that kind of thing. I'll say this about inspections; although they didn't help they didn't hurt. The main reason we don't do them anymore is because we didn't get processes in place to make them more automatic.

You might ask, "Once we're fixing our bugs first, why bother having an alpha at all? Why not just add zero-defect features all the way up to our final ship date?"

I found on my last project that even with ruthless fixing-of-bugs-first, even with in-house testing, we still had more bugs than we could handle post-alpha. But supposing the system worked as intended, and you hit alpha with no high-priority bugs. Then you're a big fat winner. You can now spend the next few months fixing low-priority bugs, polishing gameplay, even introducing low-impact features, and acheiving a level of quality rarely seen in the videogame world.

Of course, if anybody has more ideas on how to keep the bugs down, I'd love to hear them.

We take videogames seriously

We just pooled our money to have Chad Proctor drink a cup of "Special Garlic Sauce." He didn't throw up. These things happen during E3 crunch.

We've been playing a lot of Universalis lately. No, it's not a computer game. But I've had a hell of a lot more fun playing it than I have had playing computer games lately. I'm becoming addicted, in fact. It sounded like a dumb idea when I first heard about it; "You tell a story together?" I assumed it was bound to end up with results like this. It turns out that the game provides enough structure to usually prevent that from happening. (We've played half a dozen times now and only once did the story completely derail. And we still had a good time.) And as long as there are enough people playing with enough ideas to keep things rolling...(five seems like enough; four is pushing it)...it's great stuff. It's the Karaoke of storytelling; the narratives we've produced would probably be pretty painful for anyone else to hear, but the enjoyment we got producing them was much greater than the enjoyment we'd typically get from being on the receiving end of mass market narratives. (I admit it. I like Karaoke. I used to think I was too cool for it, but I'm getting old.)