Posts in the Software Development category

I've just gotten to a good place at work today. Finally, a long project is approaching its conclusion. So I've been thinking about what I know about project management, and working from my experience on tasks like this (and watching others tackle them) I came to some conclusions about how this kind of process works.

What is a broad task? Or a narrow one?

A broad task is one that cannot be easily broken down into pieces to attack. That's not to say you can't try; it's just that there are large numbers of unknowns at the start of the process.

A narrow task is one that is easily defined, narrow in scope (ie. tackles a very specific, small problem), and that is reasonably accurate to estimate.

Examples of broad tasks include any kind of integration work, any kind of porting work from one platform to another, and any time you need to build a new framework.

Examples of narrow tasks are those that could be described as "small features". For example, adding the ability to drop down a color-picker and select a color in an editor would be a narrow task. Adding the ability to specify that a mesh uses 16-bit floating point values for vertex positions instead of 32-bit floating point values would also be a narrow task.

The Graph of Progress over Time

Here is a graph of what a broad task looks like to the engineer doing it, and to the outside world, in terms of progress made. It's slightly scary, but that's ok, we're going to go through it in detail.

We're focusing here on the experience of the engineer themselves (because it's important to understand what they're going through as they do the work in question), and the experience of their managers/investors... the management layer which has a strong interest in getting the task done, but might not care quite so in-depth about how it's achieved. Basically, the people with financial responsibility for the work. The ones who have to explain to investors what's going on - or indeed, the ones who have to go to investors and ask for more money for the project to continue on a regular basis.

Visible Progress

In the above diagram, the yellow body of the graph is the amount of visible progress. This is what everyone else sees - managers, executives, people on other teams - unless you're providing detailed status updates. And by detailed, I mean, showing what you did every single day. It's unfortunate, but perception really is king - you will hang or fly based on what other people can see you doing.

With broad tasks, it's crucial therefore to make sure people know what's going on - they won't understand if you're not making visible progress - all they have to measure you by is the visible progress alone.

Expected Activity

The red dashed line (the straight one, for those of you who are color-blind, and I apologise for my poor color scheme if you are) is the expected amount of progress. It's when this line doesn't match the visible progress that people get ... well... kind of nervous.

In a perfect world, this line is what your output would be. But people aren't machines. Even taking unforseen circumstances (life intruding on work, discovering that the task is a bottomless pit that no one understood until now) into account, no-one ever works like this. Let's ignore the ebb and flow of up and down days, and assume that this line is an average. Even so, the shiny happy engineer will not match this line (unless they're an engineer in the category I like to call Bulldozer). They're human, and there's a stimulus-response thing you have to take into account here.

Either way, for planning purposes (both because you can't really plan any other way, except by padding... and also because they're not the ones doing the work, so they're removed from the immediate situation), this is what management normally expect of their warm bodies.

Actual Activity

This is the green, curvy dashed line. And it's curvy. Boy it's curvy. And it's a little badly drawn (view it more of an emotional line; if I'd done it completely right, it would always slope at least imperceptibly upwards; at worst, it'd be horizontal).

This line is the actual productivity and progress of an engineer on a given broadly scoped task. Why is it shaped like that, you ask? Well, let's break it down.

The Lifecycle of a Broad Task from an Engineer's Perspective

Ramp Up

Any broad task can be pretty reasonably expected to be at least somewhat new to the engineer performing it. A certain amount of book-keeping and surveying has to be done before the engineer can even begin, because you need to figure out where to start, and where you're going.

This part of the curve is a gentle, shallow ramp (far left of the graph). It soon peaks, however, as the work gets going.

Fatigue

Next up is our first dip. Literally, at this point, fatigue sets in on the developer. They've been bashing their head against their desk trying to break the task down into bits and pieces they can manage. They've been working a while, and churning on the problem, but at this point their work provides little gratification. Their own perceived progress level is excessively low compared to the amount of work left on the task - and that worsens the problem, because now they're feeling like they're useless and they still have a huge task ahead of them.

Disheartened, they step off the gas pedal, until either someone steps in and prods them, or they get their own internal second wind. They're in the middle of a death march at this point, and they know it.

First Flush of Success
Hey, what do you know? That curve does turn back up (unless the engineer in question quits - which, if it's a huge task with no end in sight, does happen). Why?

Broad tasks are basically - as far as perceived progress goes - exponential. You're tackling problems all over the place at first, and nothing works. Then, as you keep ploughing on through the problems, you start to gather steam as you get enough of the foundation problems fixed to start tackling specific problems.

The problem with foundation issues is that ultimately, you have to fix them. You can't put them off. If you don't fix them, nothing works. But fix enough of them, and that picture changes. You can start tackling specific problems. And specific problems have a number of great qualities:

• They're narrow (aka specific). You can look at them, and see the shape of them. They fit in your brain.
• They're finite in scope.
• They're understandable.
• They're typically jenga blocks in reverse. Take them out, and whole systems come online.

Do the problems actually change at all? No, you're actually fixing specific problems the whole time. It's just that they were buried in a sea of other problems, and so they were masked, and you were also thinking about all the other problems as well.

Think of it as a huge party, where hundreds of people are shouting as loudly as they can, and everyone's standing shoulder to shoulder... and not only that, but you're the host, so your responsibility is to go and talk to every single person - even if you can't even squeeze through the crowd. Once you get a little breathing room and throw a bunch of people out, you can start having a normal conversation, and spend some quality time with your guests.

Power Curve

Here's the fun part. OK, so you're starting to thin out the problem "herd" and you've finally hit the point where not only do you know most of the systems you're working on pretty well at this point (ie. you know the mental geography of the problem you're fixing), but the herd is really thinning out. You've reduced the sea of problems down to a few rivers.

And all of a sudden, your activity level goes up, because you're making real progress. At least, you can see that you're making it.

Back to the cocktail party analogy - you're looking for people who need a fresh drink. Much easier to spot when there's only about 10 people in the room, instead of 100. Not only that, but you start noticing similarities between them - maybe three people want martinis, so you can do all of those in one go and save yourself some time.

All of a sudden, you're fixing problems, and systems start to come online. Maybe they don't work yet, but you know what you have to do to get them to work. And you know what? That's a great feeling.

In the Zone

This is the great part. All of a sudden, you've reached the point where you're making regular fast past progress. The brunt of the task has been done, and now you're at a point where you can get a good handle on your work. Here's the cool bit.

This is where the externally visible progress part of the graph takes off like a rocket. You've already slipped out of your funk, and have been grooving on the system for a while now. And with your happiness at the progress you've made, and all the positive feedback that gives, you're on a bit of a high. So your output increases.

But not only that... shortly after this point not only have you nailed enough of the problems to the wall that you're happy about what you're going, and can make targetted attacks, but at this point enough of the foundation work has been laid that the problems that remain aren't completely deadlocking your work from... well... working. All of a sudden whole systems are coming online with a few annoying hangnails here and there where things crash or don't work right. The important thing though is that until this point those systems didn't appear to work at all.

It's like an exponential curve. You've got to get past all of the long flat part until you hit the knee, and then it shoots off like a rocket.

Suddenly, to the outside observer, you're making continued, amazing, daily (or even hourly!) progress. Has anything changed?

Not really. You hit that point personally about a week before, but no one could see what the fuss was about. But the slope of your expected activity curve is now looking mighty shallow compared to the visible progress. Vindication! All those hours where people were wondering if you should be considering an alternative career path are now worth the pain.

And what does that do?

It bumps your activity up even further. Now you're jazzed, because not only can other people see what you knew you'd been doing all along, but the task is now starting to look easy to you. The visible progress curve lags the actual progress curve; at this point, you're down to a cocktail party with about 3 people, and you're serving the coffee and brandy.

Back-Patting

It's inevitable that an engineer after a long, arduous task, is going to indulge in a little back-patting. After all, if this was a race, then they've just been sprinting non-stop - which was a wonderful change of pace after trying to run through the middle of a swamp.

The only issues left to fix are pretty much in two categories - minor ones, which aren't preventing immediate progress, or huge ones which were discovered as the work went along, and have been worked around for now, and need some serious TLC in the near future.

But not right now. Because right now, they need a bit of a rest.

This phase is pretty short. It comes as the amount of issues remaining drops to only one or two. The top of the visible progress power curve is reached right at this point, and begins to slope down again - but now it's pretty much consistent progress going forward.

Refractory Period

Back patting is over, the engineer is a little mentally exhausted still (the low after the high of getting as far as they did). Now starts the ramp back up to regular productivity levels (the expected line now does have true meaning, because presumably we're back to narrow scoped tasks).

If you're going to give them some time off for good behavior, now is the time. They've milked their praise, and that little extra bit of buzz is wearing off. So now that they've done their work, and given their bows to their peers in the audience, let them go off stage gracefully and recoup their physical and mental energy.
And lo and behold... the task is done. It was a rocky road. And some rock stars were discovered.

The Secret

Some people can get past the initial curve by discipline and stiff upper lips alone (the aforementioned Bulldozers).

If you don't want to get fired, fight that visible progress curve by providing regular progress updates - never go "dark".

And the biggest secret of all?

All tasks work like this. Even the little narrow focused ones. It's just that it's over such a short period of time that you might not even notice the ups and downs, and the relative durations of each of the phases will change. About the only time it differs is if you've done something very similar many times before, in which case you might be able to avoid most of the doldrums at the start.

Even big, multiperson tasks work like this too (eg. Entire Projects). Different people in different disciplines might be at different points in the curve, but large projects do work like this.

Project Managers, Lead engineers (call them what you will) try to fight the visibile progress curve with a number of tools - milestones being one of them - for a number of reasons. Executives and financiers don't like it when a project isn't making visible progress at a consistent rate. Potentially more importantly, people can feel the ebb and flow of a project, and this can affect their productivity. Best to keep a good shiny face on things, and continuous displays of concrete progress will do that for the team - and that's good for the health of the team.

The only trick is, don't ever, ever, ever jeopardize the foundation of your project (you know, that long shallow part) solely to satisfy the need for visible progress. Because that will come back and bite you in the ass, big time, as that foundation you skimped on crumbles around you later. If you must muck around with that part, make sure you have a plan, everyone agrees on that plan, and you have a written document which specifies how and when you're going to go back and fix things up. It'll be more painful than doing it right up front, but that's the trade off you make.

More on that in a later post.

I'll be at Gamefest for the next couple of days... w00t :)

Nice change of pace... lots to learn... friends to reune with. (OK, so that's probably not a word, but how else would you turn reunion into a verb? :D)

Should be fun.

A while back I wrote an article on CodeProject about a bi-partite circular buffer algorithm I came up with to handle asynchronous network IO. (It's also useful for other things - pretty much any scenario where you have to pass data in contiguous blocks to other APIs, yet you don't know exactly how much data you're going to be passing at any time).

Well.. I just came across this blog post from someone:

A Pure Programmer
I read an article about circular buffer and related code, which written by Simon Cooke. It’s very good. I never heard of Simon Cooke before. I was moved by the last words in his article: “If you do find it (the code he write) useful, or use it in any of your code, all that I ask in return is that you drop me an email and let me know how the code is being used. It’s nice to know that it's out there, alive, and doing cool things."
Oh,what a pure programmer!

Awww... bless. It warms my heart that someone appreciates my work, it really does.

I must admit, I was rather proud of that little piece of code. It worked out to be pretty fast too - the only way to do anything faster would have been to use the virtual memory mirroring technique I laid out in the article - but unfortunately, that doesn't work on some architectures. (I'm looking at YOU, XBOX 360). Not sure if it's multiproc safe on other systems too. Damn cache coherency, I stab at thee.

Followup:

So I got a bit narcissistic and did a Google search on the Bip Buffer... and lo and behold, people are using it. One guy's looking at it as a way of performing least-wear writing to flash memory (now that's a cool application I never even thought of!). So glad this code's getting some use!!!

I'm often amazed by people who defend CSS as an amazing technology for layout. I get where they're coming from - I mean it is a pretty reasonable way to handle styles (font stuff, mainly), and it has a laudable goal - namely to separate content from the layout. But as a technology for marking up layout? It's about the lousiest thing I can think of. It's painful. it's a chore.

CSS3.0 is starting to address some of these issues (border layout handling, for example, is something that we should have had since the start - and by start, I mean since Netscape 1.0), but it still needs to be backwards compatible with the older CSS functionality, which sucks.

But don't take my word for it. John Nagle doesn't like them either. (And yes, that's right, that's John Nagle. As in Nagle Algorithm). Here's a post I found from him on this blog:

You’re absolutely right.With Dreamweaver 3 and tables, it wasn’t necessary to look at HTML to lay out a page. With Dreamweaver 8 and CSS, the page designer must understand CSS, HTML, and probably Javascript. That’s was a big step backwards.

The CSS system is just too programmer-oriented. And I’m a programmer. (Programmer as in MSCS from Stanford, the Nagle algorithm in TCP, inventor of ragdoll technology, real-time robot vehicle control, not programmer as in “writes some Perl”. And my first web site went up in 1995.) It’s not that CSS is hard; it’s that CSS is bad.

CSS is, simply, a badly designed layout system. Even the rather simple system in Tk which lays out dialog boxes and windows is better. Tk is a nested-box system, but both “pack” (like CSS “float”) and “grid” (like tables) layouts are available in the same system. This is enough to handle most cases. Which “float” and “clear” are not. Page layout is forced to fall back on absolute positioning far too often.

The clever way to do layout would have been with a constraint system. Each box has four edges and four corners, and it would be possible to bind corners and edges to create any desired relationship between boxes. This is something one could express easily in a click and drag graphical tool. Want three columns the same height? Tie their adjacent bottom corners together.

Want to fill the page? Tie the outside corners to a page edge. Ten minutes to explain to an artist. Advanced use would involve priorities on constraints, so if something had to give in “fluid design” as the page size or type size changed, you could pick what gave first. (This could be extended to allow curved boundaries, even splines, but that might be overdoing it.)

The browser would have to have a constraint engine to resolve all the constraints, but there are known solutions to that problem.

Too many people drank the Kool-Aid on CSS. It’s just not that good a technology.

...

The worst problem with DIV-based layout is that the layout system is too weak. There’s no form of “grid” layout. There’s no way to relate a DIV to anything but its predecessor, its parent, or an absolute position. The system is just too dumb. That’s why people have to stand on their head just to get three columns to work.Tables actually are a better designed layout system. Table layouts allow table cells which span multiple rows and columns. If all tables could do were simple grids of cells, the CSS approach might make sense, but tables are more general than that. And they’re well supported in Dreamweaver.

The fundamental limitations of DIV-based layour are obscured by an excessive number of attributes and the occasional use of Javascript when the attributes aren’t enough. But underneath, the fundamental approach is just too weak.

If CSS had a grid capability, it wouldn’t be so bad. But it doesn’t.

So there you have it. CSS sucks.

I'm thinking about an alternative solution for some of the problems... If I get time I'll post it up.

OK, I know, a bit egocentric to name a law after yourself, but here goes. It applies in all kinds of places, but it's really for the most part a Software Development law.

Architecture is persistent.

The moment you sit down and start writing code, you're laying down the architecture for your software. It doesn't matter what you write, it has infrastructure. All code is infrastructure, with other code that does something to the data. Most of the code you will ever write is spent navigating that infrastructure to gather data and perform operations on it, and then store that data out.

This gives us a number of interesting little results that you see in the real world all the time:

Once you write some code, the architecture you decided on sticks around.

Seriously. The more code you write, the more the architecture you specified becomes embedded in every piece of code you write. You can't help it - it's part of the code's DNA. Every choice you make will pervade everything your code does forevermore.

For example... You write a large piece of code which has a scripting system which is used for everything - header generation, you name it. Script based objects are garbage collected.

Now, later, you decide "Hey, wait a minute... this GC is slowing me down... the script language isn't completely fleshed out so it's hindering me, and it's not suitable for all of the tasks I envisioned at the start".

Tough cookies. At this point, a year later, you now have so much code written based on this architecture, that even if you find ways around it, your code will still taste of the existing architecture. You can refactor all you want, but until you deliberately start a second branch of code that is an isolated and sterile environment, that only talks to the original code through very hard specified APIs - kind of a software firewall - you'll have that architecture still. Even if you rip out every single line of code that uses the original architecture, unless you effectively develop the new code in a vacuum, your code will still have that architecture embedded deep inside it.

People who have worked on large projects involving 3rd party code which doesn't work as advertized should recognise this one.

If you don't specify an architecture, you get one anyway.

It might not be the architecture you want, but it's certainly an architecture. The problem is that the code will generate structure as it goes, and that structure is architecture. It'll persist just as much as the original architecture, and all you can hope for is that this hackery wasn't part of your main code path. (Which means you might, just might, be able to snip it out).

Never write code ad hoc. Always at least spend a moment thinking about the architecture of the code you're writing. If you're dealing with 3rd party code, try to analyze the architecture of their system before committing your own code to it - misunderstanding their design is bad, as you'll end up creating incompatible architecture around it, and doubling your work - you'll have wrapper code which does more than it needs to, and then that'll spawn its own kind of architecture.

If you're dealing with 3rd party code which hasn't got any architecture, and was developed ad hoc, you're kind of screwed. It'll be difficult to get the architecture to sit in your brain.

Most software development - from a human perspective - is the building of abstract models of how something works. The easier this is for someone, the more easily used that code will be. If your architecture is clean, easy to understand, logical and consistent, it'll sit well in someone else's brain because the number of details they need to remember goes down rapidly. They don't have to remember all of the function names in your code, or all of the nitty gritty details. All they need to remember is roughly what shape that code has (its model), and how that relates to other similar code they've seen, and other design patterns they've experienced.

Take WxWindows. It shares a number of architectural similarities with ATL Windowing code, WTL, MFC and .NET Windows Forms. (Personally, I don't like working with it as much as any of the aforementioned libraries because of a variety of reasons, but that's besides the point). This means that anyone with experience in Win32 UI development should be able to very quickly pick up WxWindows programming. The models are the same - the architecture is similar. Basically, it's a message passing system, with event hooks, and some resource handling for the creation of controls. The function names may be different, and some of the implementation details may change, but for the most part it's an easily understood model - because it's a similar model to other environments.

Things that can make code easier to understand for other engineers tend to also be things that give a solid architectural base:

• Design your code first. Make a laundry list of the different things you're going to need to make your app, and figure out how you want to handle everything on that list. It should be as exhaustive as possible. You can always add later, but you want to be able to find any fundamental architectural pieces that you need to implement so that the rest of your code will use it consistently. You don't want any surprises.
• Try to write code from the top down. When writing any class, decide how you - as a programmer - would want to write code against that class. Write that code first. Then write the class to fit. Stub out functions if you need to (but make sure you mark them so that you know you need to finish them up).
• Write comments at the function and class level. These describe the model you're using, and separate out implementation specific details (eg. how exactly you write a serialization function) from the structure of the application (ie. the fact that it actually performs serialization in a standardized fashion).
• Once you've decided on a pattern for handling a scenario in your code, stick with it. Don't alternate patterns. For example, if you return error codes in one place, return them everywhere. Don't switch to exceptions somewhere else. If you need to use them, wrap them in a try catch at the lowest level you can, and return an error code as a result.
• Read as much code as you can. Try to find out what works well, and what doesn't. More importantly, try to understand if there was a good reason behind the way that code was written, or if it was created ad hoc.

In summary?

Other programmers aren't stupid. They tend to be smart, driven, passionate... and under tight deadlines. And the nasty side effect of tight deadlines is that architecture drops by the wayside. Try to keep your architecture as clean as you can, no matter what the deadline. The extra time you spend now will pay itself back later when you need to maintain or extend that code. And other developers who have to work on it will thank you for it.

What's more, if you have a clean architecture, it'll fit in your brain better, and you'll spend less effort. What's better than that?