This post is about designing cameras for video games, and for anyone who wants to shoot a movie. I think I’ve finally figured out why spirit levels are placed on tripods, why some people hate shaky-cam, why you need up to be up, and why dutch angles feel… well… off-kilter.
The reason has to do with your vestibulo-ocular reflex.
So what is the vestibulo-ocular reflex?
Well, you could read the wikipedia page, but basically it’s a connection between the semicircular canals in your ears, which are mini-gyroscopes which tell your head which way up you are, and your eyes.
If you tilt your head to the left, the VO reflex makes your eyes twist to the right. If you tilt your head to the right, they twist back the other way. This effect happens up to about 15⁰, at which point it stops, because your optic nerves are a big long cord, and your body would prefer to not get them all twisted up in knots by letting your eyes twist and twist around in their sockets.
Don’t believe me? Look closely at your eye in a mirror. Find a blood vessel. Then tilt your head from side to side. Notice how the blood vessel remains in roughly the same place relative to the ground while youre head moves around it? Freaky, isn’t it?
(diagram stolen from wikipedia)
This – as I’ve covered previously in another post – is also the reason why you get the spins when you’re exceptionally drunk, and also gives you the cure for it (tilt your head enough that it gets a signal that’s impossible to ignore even when your gyroscopes are slightly messed up).
But why do we have a vestibulo-ocular reflex at all? Or muscles that rotate our eyes like that? What’s the point?
That wikipedia article mentions the primary reason – to stabilize the image on the retina. That’s fine for panning left-right and tilting up and down… but why on earth would we need handling for rolls (torsional motions)?
I don’t think I’ve seen any medical papers on this, but I’m pretty sure I’ve figured it out.
The answer is that it’s a cheat. An optimization.
Your brain is optimized for gravity
Or, rather, your visual system is optimized for looking at the world with up being… well… up. You’ve got some machinery for rotating objects in your head, but it’s pretty lightweight (although very impressive!) and as a result, it’s easily overloaded.
But gravity means that up is up, and down is down, and it’s been that way since the Earth was formed. So we evolved a cheat.
We only store the full information for recognizing objects in detail when they’re in the normal orientation we expect to see them. Faces, for example, we expect to see upright – that system’s very easily fooled if you give it unexpected inputs. This is known as the Thatcher Effect.
The image on the left is Robert Downey Jr. when he was in Iron Man 3. The image on the right is Robert Downey Jr. when he was doing a shipping container full of lovely, lovely drugs.
Okay, okay (hey, I love RDJ, so please don’t take that seriously – he’s one of my favorite actors). The image on the left is the same as the image on the right; it’s just upside down on the left. If you turn your head, or if you’re using a tablet, flip it around, you can see that they’re identical.
The reason the left-hand version looks fine is because your visual system is seeing his features in their expected orientation.The one of Mr. Junior on the right looks weird because those features are completely off – but everything else is as expected. Your brain tries harder to parse the image on the right correctly, because the gross features (the blurry mess you can see when you squint) match what you’d expect for an upright person.
You don’t even have to mess around with the image itself; just stand on your head and look at someone else – it’s harder to recognize them upside down – but the Thatcher Effect exploits this effect and magnifies it.
What we have here, is the clues to the hack.
Your brain cheats, and reduces the amount of processing required to recognize an object by making sure that the image on your retina – at least as far as it can manage - is always upright. This is why it’s more difficult to watch TV when you’re lying on your side in bed too – your brain and evolution have taken a shortcut, and not bothered to learn how to rotate scenes that much. Why bother wasting expensive neurons on it, when you can hack it with a gyroscope and a couple of muscles in each eye? Other animals have similar hacks too – and the inferior and superior oblique muscles do the rotation work in them too.
How to make yourself sick
Let’s say you’re in bed, in a dark room, watching a movie or playing a video game. There are two ways to easily make yourself sick because of this hack.
The first is, with your head against the headboard of the bed, tilt your about a degree from side to side as fast as you can. You’ll soon start to feel sick, because the image you’re looking at on the screen is now tilting faster than you can compensate (or the signal from the wiggling isn’t strong enough to compensate for quickly enough).
I suspect that what’s happening is, because the room is dark enough, and you’re focusing on the screen, you’re treating it as if it’s the same as looking through a window. When you wiggle, the world doesn’t match what your gyroscopes are telling you, so you feel sick.
The other way is to make the in-game camera, or the movie camera, rotate. Translational movement (up/down, left/right) is fine. Panning is less fine, but okay as long as it’s not oscillating. Tilting is fine – again, as long as it doesn’t oscillate. But rotation around the direction into the screen? Yeah, that’s going to make people blow chunks really quickly.
So what does this mean for games?
What this means for games is that your in-game camera – even if attached to a virtual person rig for say a Gears of War style run-cycle camera – should almost never tilt away from up being up. If you do it, you’d better have a reason.
Most game developers have intuitively figured this out, but it’s still worth bearing in mind – we expect down to be at the bottom of the screen, and up to be at the top. If you subvert that trend, it’d better be for a good reason.
And for god’s sake, keep the oscillation to a minimum. Your VO reflex also handles panning and tilting motion, but you’re displaying what we’d expect to see through virtual eyeballs. We don’t have virtual semicircular canals, so keep the tilting and panning in an animated run cycle (for a first person game) to an absolute minimum as well. If possible, just resort to moving the camera in a 2D plane around a point. Or you’ll make people sick.
What does this mean for movies?
The movie guys are ahead of us on this a lot – except for the annoying, completely pointless and worrying trend for using “found footage” and shaky-cam in every scene that’s supposed to be “action” or even just plain “raw and edgy”. (Bleuch).
(Directors: What is the visual message you’re trying to send? You know that you could reach a larger audience, and turn less people off, by using a tried and trusted Steadicam rig instead, right? You do know that shaky-cam just hides the action, particularly for older viewers who have less processing power available to handle visuals?)
It used to be back in the day that cameras were put on tripods for most shots. Heck, you can even put those sticks on a dolly, and roll the camera around, but for the most part, it’s stable.
That’s because not doing it won’t give the same VO response, and you’ll make some people sick.
So use your tripod. And make sure that it’s level. Which means you want one of these on it:
This is a leveling head. Learn it. Love it. Live it.
And stop with the goddamn shaky-cam! Peoples’ brains aren’t designed to handle it.
Simon Cooke is an occasional video game developer, ex-freelance journalist, screenwriter, film-maker, musician, and software engineer in Seattle, WA.
The views posted on this blog are his and his alone, and have no relation to anything he's working on, his employer, or anything else and are not an official statement of any kind by them (and barely even one by him most of the time).