The Jamie Test - Step 11

Build Consensus

Going ahead a few steps. Chris's post - which, incidentally, seems to have doubled our readership (Go Chris! I hope your career will survive being blacklisted by Derek Smart) - got me thinking some of the same old thoughts.

I had an English teacher in high school who told us that nothing good ever came out of a committee. This woman also told us that the human body undergoes a weight loss immediately after death -- no doubt the departure of the soul -- and that she was visited by Jesus in her bathroom. Still, she isn't alone in her thoughts; a lot of people look down their noses at 'design by committee.'

I was reading the same post-mortem that Chris read, which talked about the failure of 'design by committee.' I think the wrong lesson is being learned here; much like when you're playing Texas Hold 'Em and you decide not to draw to the inside straight and, lo and behold, the inside straight comes up, and you rebuke yourself, and you say next time I'm going to draw to the inside straight. Wrong lesson. The project in question had a number of problems and 'design by committee' was not what sunk it. This was a project in which a lot of the team members knew the project was in trouble the whole time. If there was any 'design by committee' going on, either these members of the team weren't on those committees, or the committees were not effectively building consensus.

There's a number of ways committees can fail:

Wrong people making the decisions. (Wrong people in the committees.) Frequently you'll have a group of managers as the only ones making decisions and they forget to include the people who are actually going to do the work and know their jobs. So you're handing out job assignments to people who haven't bought in to those assignments in the first place. (I saw this in action with Minority Report. I happened to be in the room when a programmer didn't want to implement a design his boss was suggesting. The boss said, "The consensus is we should do it this way." I said, "It's not consensus if the programmer doesn't agree." "I meant it's the consensus of the leads," the boss said. This was one minor point that surely was not to blame for Minority Report's failure, but if every single aspect of the project was handled that way it could be a problem.) Another possibility, the client (usually the publisher) is not in on the committees either, and will try to control the project retroactively. And finally, people join the team after decisions have already been made; do you rehash all the decisions up to this point? Or do you just shove the old decisions down their throats?

Fear of hurting each other's feelings. At some point, brainstorming has got to end, and hard decisions have to be made, otherwise you will end up with bloatware, as every single person's pet feature is put into the game. It probably is the real problem that the people were complaining about in that post-mortem I mentioned; it's also a problem that we experienced on the team I work on. Our original design document outlined whole systems of features that were tangential to the focus of the game - not quite toasters for cars but headed in that direction. The only thing that got those features cut was that we ran out of time. If we'd been more vigilant in our design, we could have cut sooner, and the resources that did go into those features could have been spent more usefully.

False consensus. Drucker talks about the social experiment where a group of people, in a circle, are asked if two different lines are the same length. Every member of the group but one is a plant; they agree that the different lines are the same. When it gets to the test subject, he almost always caves and agrees with the rest of the group. This phenomenon is what allows a whole group of people to collectively agree on the same bone-headed decision.

Not the best ideas The ideas that go in are not the best ideas but the pet ideas of the most persuasive people. (Often the most persuasive people will be people in lead positions, granted a sort of unconscious bias.) According to Carnegie, persuasion has little to do with the 'rightness' of your argument, and more to do with how you argue.

Nothing gets done Spending eternity in meetings without doing any actual work. When thesis and antithesis compete, it takes a while to come up with synthesis. At the beginning of our latest project we spent half our time in meetings for the first few months. I think the time was well spent but one could argue that we'd have more to show if we just dove in and started making stuff.

That's quite a list, so you may want to switch to The Boss Decides method of making decisions. What this gets you is a trade-off: you're less likely to be bitten by the "fear of hurting other people's feelings" problem and the "nothing gets done" problem, but you're more likely to be bitten by the "false consensus" problem (a bone-headed consensus of one) and the "wrong people making the decisision" problem and the "ideas are not the best ideas but the ideas of the most persuasive." (In this case, the boss.) In my opinion, the fact that you now have someone Responsible running the show, whom you can fire when he fucks up, doesn't really make up for the fact that you've increased the likelihood of fucking up.

My suggestion is this: if you are in a leadership position, take the risk of abandoning yourself to consensus building. (It's hard. There have definitely been times where everybody except me wanted to do one thing and I insisted on another. I now feel that the correct thing for me to do in those cases was keep hashing it out until we achieved synthesis.) If you're not in a leadership position, make yourself heard and try to encourage others on your team to be heard as well.

How do you build consensus? There's a short guide here that I just found via Google. And if you want it spelled out in excruciating detail, and don't mind wacky space cadet stuff, then take a look at the McCarthy's "Decider Protocol" in their Core system here.

These systems solve items 2 through 4, and make some effort to mitigate item 5. What they do not solve is having the wrong people in the meetings. (Our programmer team rapidly achieved consensus when we proposed that we should all get more RAM. There was nobody from budgeting or purchasing in the meeting.) This is a problem I do not yet know how to solve. Must research.

If you still think 'design by committee' is a bad idea, consider this: like Chris said, the Half-Life guys did it. But they took three years to ship, you point out. Well how about this: the Deus Ex guys did it. In my opinion, Deus Ex is the biggest success story in all of recorded videogame development: quality delivered in a timely manner on a reasonable budget. From their post-mortem:

Should you get to name your character or not? A holy war almost broke out on the Deus Ex team about this. "If you can't name your character, it's not an RPG," said some. "If we don't name the character, how do we write and record compelling conversations and create a cool story?" said others. "Story isn't the point…" "Yes, it is…" and on and on and on. We compromised: we gave the player character a code name and back-story but let the player select his real name, which came into play in various ways (though never in speech).

Not a compromise at all, but synthesis. They practice consensus. And so should we.

Chris would say something like, "You're missing my point, Jamie. I never said design by committee was bad, I just said there has to be someone with whom the buck stops." I would argue that since, the world of business being what it is, buck stopping people are built into the system. Whoever your pimp is, they're going to want to know who the "one in charge" is, and that person is going to be considered responsible for the results of the entire team. What concerns me is this unhealthy fascination we have for the org chart. The system wires us to spend a lot more time thinking about who is in charge than we should. Because the system does this to us, and because at some level we like it (who doesn't want to see a nice fat "Technical Director" credit on the game they shipped?) we need to fight this tendency to actually make a true consensus happen. Don't worry about figuring out who is in charge of whom; worry about figuring out how to get the best ideas into your game.

Site Name Change May Be In Order

Saw a movie poster for League of Extraordinary Gentlemen yesterday. They're calling it LXG. LXG??? LXG?!?!?!?! What the fuck? What the fucking fuck? Looks like some marketing suit out there thinks they're making an X-Men thing. I guess after From Hell, I shouldn't be surprised by the butchering of Alan Moore's work. But God! Some people just don't Get It. I say to whoever is making that movie: Good job. You've alienated your hardcore fans and probably the casual moviegoing audience as well.

And now our thirty or so loyal readers will be associating this site with a lame movie instead of a cool comic book.

No Toaster Cars
Know what your game is trying to deliver, and measure everything against how well it helps deliver that thing. If your game is supposed to deliver a visceral feeling of mayhem, don't work on a realistic driving sim. Yes, driving sims are fun. But that's not the game you're making.

Making an appliance that toasts bread is good. But not if you're supposed to be making a car.

No toaster cars.

Conundrums

It seems that every project management platitude out there has its converse out there as well. To wit:

On Offices

People Should Have Their Own Offices

– Steve McConnell

The argument being knowledge workers need long periods of undistracted time in order to accomplish thought-intensive tasks.

People Should All Be In One Big War-Room

The argument being that communication is essential and therefore everyone should be able to always talk to everyone else.

On Scheduling

Optimistic Schedules Are Bad

– Steve McConnell

The argument being that to achieve an overly optimistic schedule you engage in risky and shoddy workmanship, creating a massive bug list that ends up lengthening your schedule beyond what it originally would have been before you got "aggressive."

Optimistic Schedules Are Good

– Steve Maguire, Jim McCarthy

Steve's argument being that tasks take the amount of time allotted for them; to make sure they take as little as possible, create an impossible schedule and rest secure in the knowledge that everyone is getting everything done "as soon as possible."
Jim's argument, from Software For Your Head, is that there are no shortages of resources; only shortages of resourcefulness. I guess we could paraphrase him by saying necessity is the mother of invention.

On Managers Doing Actual Work

The second rule of Bad Management: put yourself in as your own utility infielder

- Tom DeMarco

There are several arguments here: one is management takes time and you should do it. Another is: the people on your team probably know their jobs better than you do, so you shouldn't try to do it for them. Another is: in the role you find yourself in, you're going to be distracted on a regular basis, and therefore not going to be able to focus on doing actual work.

Just Do It or You Pet The Kitty, You Own The Kitty

- Joel Spolsky / Jim McCarthy

I'm paraphrasing here. Joel points out that multitasking is harmful, so if you see a task that needs doing, and everyone on the team is already working on something, it's better to do it yourself than to make somebody task switch. Jim McCarthy says the person who proposes something should implement it, since he's the person who gives a shit.

I'm sure there are more examples of these. In these cases, I think there are solutions where a kind of synergy between both arguments can be developed: for the offices question, we could have a central war room ringed with private offices; "your" computer is in the war-room, but when you need privacy, you can borrow one of the private offices. (I've never heard of anyone actually trying this.) An amusing side note is that on my team we do the exact wrong thing: the people who are actually doing the work are in cubicle farms or share offices, where they get maximum distraction and very little communication, whereas the leads – who should be doing most of the communicating and less actual work – get their own private offices.

And with scheduling, you can use the method from Critical Chain and Slack: schedule deliverables aggressively but have a fat safety buffer at the end of the project. In a way, the industry already does this and calls it 'alpha', although we supposedly do it as a time to fix bugs (which encourages sloppiness, IMO.)

And for that last one, I have no idea. Must think.

Jamie Test, Step 8, Part 2

Let team members estimate their own schedules

More on the scheduling thing: I thought I'd share some of our experiences working with Joel Spolsky's scheduling system.
There are limitations to his system, and while it's still the best thing I've found out there--it beats the crap out of scheduling on filecards, XP style--you have to be aware of the limitations or you can fall into some traps.

- The "Debugging" pad / buffer can allow you to break the system. Supposing I code up a feature. I use up all the hours I've scheduled for it, and it's "done". One problem: it doesn't work yet. So I keep banging my head against it, and log the time as "Debugging." This isn't a problem for the lucky few who write code correctly the first time; people like me, on the other hand, tend to write a collection of bugs, fix the bugs, and then we have software. If we log the 'bug fixing' phase as "Debugging", then we aren't measuring correctly, and the scheduling tool loses its power. The way to fix this is to make sure everybody on the team is aware of this pitfall, and doesn't call a feature 'done' until they've gotten it to the point where there are no stop-shipment bugs that they are aware of.

- The schedule does not explicitly discourage multitasking; you may find that someone is switching from task to task. This could be because different people are asking them to work on different features, or because they just got sick of working on one thing and decided to work on another. Multitasking is bad. Unless truly higher-priority items come up, let people focus on one thing at a time. It's hard to do; everyone wants everything done at once, and they will be constantly asking, "Isn't someone working on this?" and the easy way out is to say, "They are now!" and switch someone.

- People get confused by the whole 8 hour day thing; on our team, we tell them "they aren't hours, they're eighths of days"; on other teams they go to daily. If you spend a half day on something, then you elapse .5 days. A meeting might be .25 or .15.

- Joel says to keep subtasks under 24 hours. In my experience, if you tell your team this, they will frequently underestimate a task (yep, that'll take 24 hours) rather than take the time to subdivide it correctly. (You could argue that we should fix the real problem rather than hack it.) I'm happy with 40 hour granularity; that may be because our schedules tend to project out for almost a year, and we don't need the finer grain.

- How to get everyone to correctly elapse their time every day? If everyone starts on the same day, then they should all have the same number of hours elapsed at the end of every day. So at the end of the day, a manager can run a pivot table that sums everyone's elapsed hours. (Jem Maza found a clever way to automatically pivot into a different spreadsheet.) Then we e-mail that table to the team and people can see who is off. If someone is particularly bad at remembering to elapse, you break out the whip. This is important; the longer you go without remembering to elapse your schedule, the less accurate it gets.

- When someone runs out of time, and someone else still has slack, we'll move tasks across. Because resources aren't fungible, we erase the previous coder's estimate, so it doesn't bias the estimate of the next coder.

- Have a wishlist page in the schedule doc. When the whole schedule is out of slack, it's time to cut features. I cut them from the main page and paste them on the wishlist page. If some time frees up--that very rarely happens--we can move them back. When people come up with new ideas for features, they go onto the wishlist. You can look at the wishlist at the beginning of your next project.

- At certain stages of the project, if we find that someone is consistently under-estimating, we'll go in and pad their remaining estimates. This is usually followed by a spree of reassigning and cutting features.

There are a few arguments against the system: let me address them.

1) "You say this lets me self-manage but it seems more like self-micromanagement to me!"

Micro-management does not mean to manage something in excruciating detail; it means a manager is telling someone else how to do their job. You should be managing in excruciating detail; you should not be telling someone else how to do their job. The scheduling system puts the responsibility for managing the schedule where it belongs: in the hands of the person who knows the most about the problem.

2) "The time I spend scheduling I'm not actually working!"

This isn't really an argument against this system but an argument against scheduling in general. And it's not completely without validity! Scheduling is only useful up to a point, something I talk about here. But for the most part, scheduling is going to be a fact of life, particularly in the production phase of your project, so you might as well use the best system available.

3) "I can't make a decent estimate on how long something is going to take until I've worked on it some."

Scheduling takes time. We admit this. That's why we have a field on the schedule for the act of scheduling itself. If the way you want to schedule is to start working on a task for a little while, until you have enough of a handle on it to make an estimate, that is totally fine. In general, I've found that programmers are surprised at how accurate they turn out to be at the end of the project. In my experience, once estimates are averaged out over a long period of time, everyone comes in between 90% and 150% of their original estimates.

4) "How can I possibly know with that degree of precision how long my tasks are going to take?"

Precision isn't important; sometimes you're over, sometimes you're under; it all comes out in the wash.

5) "How do I know when I'm supposed to work overtime?"

When your boss tells you "have this done by Friday" and it's Thursday and you're not done you know you need to stay late. When your boss is some kind of Mr. Softie who only expects you to deliver half of your tasks early it's not so clear. I believe that some overtime is important; that no matter how good a game you can make in a forty hour work week, you can make a better one in fifty, and marginally better still in sixty.

We work overtime when there's an unavoidable external deliverable due and we're not sure the game's going to be good enough by then. So we're crunching right now (50-60 hour weeks) for E3, because we're not Sure.

Another reason to crunch is when our bug find rate is higher than our bug fix rate; during this period of time, we're losing ground, and the only way to make it up is with good old overtime.

6) "It doesn't handle dependencies!"

This is the big one, and we've definitely been bitten by it: a version is due, and it's supposed to have feature X, and there's plenty of time for each person working on feature X to get the subtasks done, but the subtasks have to be done in order. Personally, *I* don't care: I'm happy with: when person A gets to their part of feature X, and they discover they're dependent on Person B, we make feature X the highest priority for Person B, and let person A move on to feature Y. Feature X doesn't make it into the rev, and we live with it. It's my belief that, over the cycle of the entire project, the effects of these dependency chains disappear, because these dependency chains are under a month or so: mission requires character which requires animations which require model, for example. As long as we prioritize these things correctly, and push them through the factory, they'll get done before alpha. Of course, the people who evaluate your milestone deliverables may not be as casual about it as I am.