The Right Way to develop games...

From RockPaperShotgun, an interview with Kim Swift of Valve on Portal:

KS: I think it’s the process. As far as screen-testing goes, it’s usually not until the end of the movie process. We started testing as soon as we got here. And I can understand how it would destroy a movie, because it kind of did that for Narbacular Drop. We didn’t start play-testing until really the last month of our development, and by that point it’s too late to do anything about it. It’s like, “Oh, well, that’s unfortunate.” I think it’s when you start to get user feedback that’s most important. Because we’re in an entertainment industry, and if people are not entertained, then we’re not doing our job.

Emphasis mine.


Why People Read My Blog (according to Google)

According to Google's Webmaster Tools one of the more popular search terms that leads to a click on my blog ( is “Sandpaper porn”

Sandpaper porn!?!?!?!?!?!

I’m at a total loss for words. Here’s the funny ones…

  • Milkshake racing game (Race that milkshake!)

  • Bruised coxix (Ow!)

  • Sandpaper porn (Double triple quadruple ow!)

  • 2005 fully clothed sex (Not as much fun as 2007 fully unclothed on the balcony sex)

  • British dish mush (Hey, my cooking’s not that bad)

  • Accidental pee drunk (we’ve all been there…)

  • Diuretics peeing a lot (Yes, yes, they do…)

  • Musclewomen porn (Lots of muscular women fans out there) (Very popular in India and Turkey, apparently)

  • Muscle woman porn (Some of them don’t like groups though…)

  • Pain receptors drunk (The best way for your pain receptors to be)

… and a couple of poignant ones:

  • “How to commit suicide using Unisom” (I’ll leave that one alone)

  • “Scientist Lost” (Like Milton’s Paradise Lost, but geekier)

  • “Books are good enough, but are a poor substitute for life”

I get a scary number of hits for sandpaper porn. Must be some kind of meme going around.

For those interested:
2005 Fully Clothed Sex takes you a follow-up post I wrote on a post of mine describing the Rules Behind Teen Movies (Not Quite Teen Movies). I’ll let you figure out the others yourself…

    Of course, this is all going to horribly skew things now, because I've made a recursive reference to the search terms in my blog. Yikes!


    The Visible Progress of Broadly Scoped Tasks

    I've just gotten to a good place at work today. Finally, a long project is approaching its conclusion. So I've been thinking about what I know about project management, and working from my experience on tasks like this (and watching others tackle them) I came to some conclusions about how this kind of process works.

    What is a broad task? Or a narrow one?

    A broad task is one that cannot be easily broken down into pieces to attack. That's not to say you can't try; it's just that there are large numbers of unknowns at the start of the process.

    A narrow task is one that is easily defined, narrow in scope (ie. tackles a very specific, small problem), and that is reasonably accurate to estimate.

    Examples of broad tasks include any kind of integration work, any kind of porting work from one platform to another, and any time you need to build a new framework.

    Examples of narrow tasks are those that could be described as "small features". For example, adding the ability to drop down a color-picker and select a color in an editor would be a narrow task. Adding the ability to specify that a mesh uses 16-bit floating point values for vertex positions instead of 32-bit floating point values would also be a narrow task.

    The Graph of Progress over Time

    Here is a graph of what a broad task looks like to the engineer doing it, and to the outside world, in terms of progress made. It's slightly scary, but that's ok, we're going to go through it in detail.

    Graph of engineering progress over time on a broad task

    We're focusing here on the experience of the engineer themselves (because it's important to understand what they're going through as they do the work in question), and the experience of their managers/investors... the management layer which has a strong interest in getting the task done, but might not care quite so in-depth about how it's achieved. Basically, the people with financial responsibility for the work. The ones who have to explain to investors what's going on - or indeed, the ones who have to go to investors and ask for more money for the project to continue on a regular basis.

    Visible Progress

    In the above diagram, the yellow body of the graph is the amount of visible progress. This is what everyone else sees - managers, executives, people on other teams - unless you're providing detailed status updates. And by detailed, I mean, showing what you did every single day. It's unfortunate, but perception really is king - you will hang or fly based on what other people can see you doing.

    With broad tasks, it's crucial therefore to make sure people know what's going on - they won't understand if you're not making visible progress - all they have to measure you by is the visible progress alone.

    Expected Activity

    The red dashed line (the straight one, for those of you who are color-blind, and I apologise for my poor color scheme if you are) is the expected amount of progress. It's when this line doesn't match the visible progress that people get ... well... kind of nervous.

    In a perfect world, this line is what your output would be. But people aren't machines. Even taking unforseen circumstances (life intruding on work, discovering that the task is a bottomless pit that no one understood until now) into account, no-one ever works like this. Let's ignore the ebb and flow of up and down days, and assume that this line is an average. Even so, the shiny happy engineer will not match this line (unless they're an engineer in the category I like to call Bulldozer). They're human, and there's a stimulus-response thing you have to take into account here.

    Either way, for planning purposes (both because you can't really plan any other way, except by padding... and also because they're not the ones doing the work, so they're removed from the immediate situation), this is what management normally expect of their warm bodies.

    Actual Activity

    This is the green, curvy dashed line. And it's curvy. Boy it's curvy. And it's a little badly drawn (view it more of an emotional line; if I'd done it completely right, it would always slope at least imperceptibly upwards; at worst, it'd be horizontal).

    This line is the actual productivity and progress of an engineer on a given broadly scoped task. Why is it shaped like that, you ask? Well, let's break it down.

    The Lifecycle of a Broad Task from an Engineer's Perspective

    Ramp Up

    Any broad task can be pretty reasonably expected to be at least somewhat new to the engineer performing it. A certain amount of book-keeping and surveying has to be done before the engineer can even begin, because you need to figure out where to start, and where you're going.

    This part of the curve is a gentle, shallow ramp (far left of the graph). It soon peaks, however, as the work gets going.


    Next up is our first dip. Literally, at this point, fatigue sets in on the developer. They've been bashing their head against their desk trying to break the task down into bits and pieces they can manage. They've been working a while, and churning on the problem, but at this point their work provides little gratification. Their own perceived progress level is excessively low compared to the amount of work left on the task - and that worsens the problem, because now they're feeling like they're useless and they still have a huge task ahead of them.

    Disheartened, they step off the gas pedal, until either someone steps in and prods them, or they get their own internal second wind. They're in the middle of a death march at this point, and they know it.

    First Flush of Success
    Hey, what do you know? That curve does turn back up (unless the engineer in question quits - which, if it's a huge task with no end in sight, does happen). Why?

    Broad tasks are basically - as far as perceived progress goes - exponential. You're tackling problems all over the place at first, and nothing works. Then, as you keep ploughing on through the problems, you start to gather steam as you get enough of the foundation problems fixed to start tackling specific problems.

    The problem with foundation issues is that ultimately, you have to fix them. You can't put them off. If you don't fix them, nothing works. But fix enough of them, and that picture changes. You can start tackling specific problems. And specific problems have a number of great qualities:

    • They're narrow (aka specific). You can look at them, and see the shape of them. They fit in your brain.
    • They're finite in scope.
    • They're understandable.
    • They're typically jenga blocks in reverse. Take them out, and whole systems come online.

    Do the problems actually change at all? No, you're actually fixing specific problems the whole time. It's just that they were buried in a sea of other problems, and so they were masked, and you were also thinking about all the other problems as well.

    Think of it as a huge party, where hundreds of people are shouting as loudly as they can, and everyone's standing shoulder to shoulder... and not only that, but you're the host, so your responsibility is to go and talk to every single person - even if you can't even squeeze through the crowd. Once you get a little breathing room and throw a bunch of people out, you can start having a normal conversation, and spend some quality time with your guests.

    Power Curve

    Here's the fun part. OK, so you're starting to thin out the problem "herd" and you've finally hit the point where not only do you know most of the systems you're working on pretty well at this point (ie. you know the mental geography of the problem you're fixing), but the herd is really thinning out. You've reduced the sea of problems down to a few rivers.

    And all of a sudden, your activity level goes up, because you're making real progress. At least, you can see that you're making it.

    Back to the cocktail party analogy - you're looking for people who need a fresh drink. Much easier to spot when there's only about 10 people in the room, instead of 100. Not only that, but you start noticing similarities between them - maybe three people want martinis, so you can do all of those in one go and save yourself some time.

    All of a sudden, you're fixing problems, and systems start to come online. Maybe they don't work yet, but you know what you have to do to get them to work. And you know what? That's a great feeling.

    In the Zone

    This is the great part. All of a sudden, you've reached the point where you're making regular fast past progress. The brunt of the task has been done, and now you're at a point where you can get a good handle on your work. Here's the cool bit.

    This is where the externally visible progress part of the graph takes off like a rocket. You've already slipped out of your funk, and have been grooving on the system for a while now. And with your happiness at the progress you've made, and all the positive feedback that gives, you're on a bit of a high. So your output increases.

    But not only that... shortly after this point not only have you nailed enough of the problems to the wall that you're happy about what you're going, and can make targetted attacks, but at this point enough of the foundation work has been laid that the problems that remain aren't completely deadlocking your work from... well... working. All of a sudden whole systems are coming online with a few annoying hangnails here and there where things crash or don't work right. The important thing though is that until this point those systems didn't appear to work at all.

    It's like an exponential curve. You've got to get past all of the long flat part until you hit the knee, and then it shoots off like a rocket.

    Suddenly, to the outside observer, you're making continued, amazing, daily (or even hourly!) progress. Has anything changed?

    Not really. You hit that point personally about a week before, but no one could see what the fuss was about. But the slope of your expected activity curve is now looking mighty shallow compared to the visible progress. Vindication! All those hours where people were wondering if you should be considering an alternative career path are now worth the pain.

    And what does that do?

    It bumps your activity up even further. Now you're jazzed, because not only can other people see what you knew you'd been doing all along, but the task is now starting to look easy to you. The visible progress curve lags the actual progress curve; at this point, you're down to a cocktail party with about 3 people, and you're serving the coffee and brandy.


    It's inevitable that an engineer after a long, arduous task, is going to indulge in a little back-patting. After all, if this was a race, then they've just been sprinting non-stop - which was a wonderful change of pace after trying to run through the middle of a swamp.

    The only issues left to fix are pretty much in two categories - minor ones, which aren't preventing immediate progress, or huge ones which were discovered as the work went along, and have been worked around for now, and need some serious TLC in the near future.

    But not right now. Because right now, they need a bit of a rest.

    This phase is pretty short. It comes as the amount of issues remaining drops to only one or two. The top of the visible progress power curve is reached right at this point, and begins to slope down again - but now it's pretty much consistent progress going forward.

    Refractory Period

    Back patting is over, the engineer is a little mentally exhausted still (the low after the high of getting as far as they did). Now starts the ramp back up to regular productivity levels (the expected line now does have true meaning, because presumably we're back to narrow scoped tasks).

    If you're going to give them some time off for good behavior, now is the time. They've milked their praise, and that little extra bit of buzz is wearing off. So now that they've done their work, and given their bows to their peers in the audience, let them go off stage gracefully and recoup their physical and mental energy.
    And lo and behold... the task is done. It was a rocky road. And some rock stars were discovered.

    The Secret

    Some people can get past the initial curve by discipline and stiff upper lips alone (the aforementioned Bulldozers).

    If you don't want to get fired, fight that visible progress curve by providing regular progress updates - never go "dark".

    And the biggest secret of all?

    All tasks work like this. Even the little narrow focused ones. It's just that it's over such a short period of time that you might not even notice the ups and downs, and the relative durations of each of the phases will change. About the only time it differs is if you've done something very similar many times before, in which case you might be able to avoid most of the doldrums at the start.

    Even big, multiperson tasks work like this too (eg. Entire Projects). Different people in different disciplines might be at different points in the curve, but large projects do work like this.

    Project Managers, Lead engineers (call them what you will) try to fight the visibile progress curve with a number of tools - milestones being one of them - for a number of reasons. Executives and financiers don't like it when a project isn't making visible progress at a consistent rate. Potentially more importantly, people can feel the ebb and flow of a project, and this can affect their productivity. Best to keep a good shiny face on things, and continuous displays of concrete progress will do that for the team - and that's good for the health of the team.

    The only trick is, don't ever, ever, ever jeopardize the foundation of your project (you know, that long shallow part) solely to satisfy the need for visible progress. Because that will come back and bite you in the ass, big time, as that foundation you skimped on crumbles around you later. If you must muck around with that part, make sure you have a plan, everyone agrees on that plan, and you have a written document which specifies how and when you're going to go back and fix things up. It'll be more painful than doing it right up front, but that's the trade off you make.

    More on that in a later post.


    subscribe via RSS