When I entered the game industry fifteen years ago, there were two standard ways to draw shadows:  blob shadows and stencil shadows.  (There was a third option as well:  light maps, as popularised by Quake 2.  But I’m not going to cover those since they were used for handling static lighting, not for real-time dynamic effects.)

Blob Shadows

Blob shadows are a very simple concept.  If we have a moving object that we want to have appear to cast a shadow, all we have to do is to draw a sprite on the ground beneath it.  This sprite was typically an indistinct, blurry oval, roughly matching the general size of the object it was “shadowing”.  Its texture is where the term “blob” came from.  In this shot, you can see a robot with a blob shadow.  (That blob shadow is substantially larger than I would ever make it, personally.  At this scale, it’s far too easy to tell that it’s just a simple sprite).

A tricky point about implementing blob shadows is making the shadow appear correctly over a ground surface;  making it draw on top of an undulating terrain is a surprisingly tricky problem.  It’s very easy to end up with a shadow sprite that clips through the ground or floats above it, or is visible through other objects between the shadow and the camera.  Since the shadow is basically just a big flat sprite, you can’t reliably cast the shadow onto an extremely bumpy surface, or onto another object.  You’re basically stuck with just a flat circle on a (more or less) flat ground.

Fifteen years ago, these were the usual way of putting dynamic shadows on movable objects in a game.  As games became more sophisticated, we started finding ways to disguise how simplistic these shadows were.  First by scaling the blob as the character animated (so as the character moved his leg forward, the circular shadow would stretch forward into an oval shape), and then by actually giving each character several different blob shadows, each projected down from a different part of the character’s body.  This made the shadow seem to vaguely move along as the character animated in a much better way than the simple scaling approach.  In Transformers, we gave each enemy robot (I seem to recall) four or five blob shadows, each anchored to a different part of the robot’s body.  When rendered, these separately circular blob shadows appeared to merge together into a single amorphous blob that moved in a way which didn’t draw the player’s attention as being circular and static.

This technique is still around and being used today in new games.  Just a few months ago I implemented it for use in an (unreleased) iPhone racing game, where we used vaguely car-shaped blob shadows under the cars.  With some very sophisticated maths to try to make the shadow mimic the car’s rotation in mid-air, and projecting down onto the race track surface according to an angled light source and angled track.  But fundamentally, if you ignore the fancy math for scaling and orienting it to the ground surface, it was still just a single blob shadow;  just a flat sprite with a pre-drawn texture on it.

I seriously considered using blob shadows for MMORPG Tycoon 2.  It certainly would have been faster to implement than what I’ve actually done.  But all the foliage in the game would have caused problems, since the shadows would have sorted underneath the foliage and been almost invisible in many situations.  (Even in modern games you occasionally see this problem, blob shadows vanishing under ground clutter)

Stencil Shadows

Also called “shadow volumes”, stencil shadows were the high-quality alternative to blob shadows.  They’re not used as much today as they used to be, but it’s still an interesting technique.

The basic idea of stencil shadows is to computationally find the silhouette of an object from the light’s point of view, and then extrude that silhouette through space.  This extruded object is the shadow cast by the object.

You then render this shadow object using a “stencil buffer”, and use the resulting stencil data to determine for each pixel in your screen image whether that pixel is inside or outside of the shadow volume.  I worked on several games which used this sort of system.  Usually this approach was only used for the shadow on the player character, or sometimes large boss enemies;  it was really too CPU intensive for us to be able to afford to do it for all characters on-screen.  And even when we did do it, we would use simplified versions of the visual models;  usually just a series of boxes or elipses roughly approximating the character’s shape.  Finding the silhouette and building renderable extruded geometry every frame was simply too expensive.

Another interesting point is that because of the way that stencil buffers work, the shadow was always hard-edged;  each pixel was either in-shadow or out-of-shadow, so shadows couldn’t feather or soften with distance.

To do this, you had to render the shadow volume with a special shader, and you had to render any object which might receive a shadow with another special shader.  This was always more expensive than normal rendering, so quite a lot of effort often went into determining precisely which triangles might receive a shadow, just for the purposes of figuring out which bits of world geometry needed to have the expensive shaders enabled.

Fun note:  The further your shadow geometry is extruded, the longer it takes to render, both because it touches more pixels, because it will more often have to go through the expensive geometry clipping process, and also because it potentially hits more world geometry, thus requiring more stuff to render using the expensive shadow-receiver shader.  But if your renderable geometry didn’t actually hit a shadow receiver (the ground underneath the character, for example, because the character was high up in the air from a jump or something and the shadow geometry wasn’t extruded far enough downward to reach the ground), then no shadow would be visible at all;  the shadow would appear to suddenly pop into being as soon as the shadow caster moved within range.  Most games tried to hide this pop-in by making the extruded shadow geometry be built like a spike;  smoothly scaling down into a single point — kind of the opposite of the shadow volume being extruded from the torus in the image above.  If you look at games from the early 90s, you’ll occasionally see games which do this;  you can tell because you’ll see that shadows get smaller as objects move away from their cast shadows, instead of getting larger.

To my knowledge, no modern games use stencil shadows any longer.  And that’s mostly because they’ve been completely replaced by the new favoured system(s): shadow maps, which are better in almost every way.

Shadow Maps

As hardware has advanced, the power of graphic cards and rendering speeds have accelerated far faster than regular CPU processing.  As a result, we’ve started to push all sorts of processing over to graphic cards whenever possible, since the graphic cards often have a lot more processing time available.  Shadow maps are one of those cases.

The basic idea of shadow maps is to actually render our scene twice:  once from the point of view of the light, and then again from the point of view of the camera.  By comparing what’s visible in the two shots, we can tell what is in shadow and what is visible from the point of view of the light.

In this screenshot I’ve put a copy of the light’s render of the scene in the top right corner.  A few things to point out about it:  the light doesn’t care about colors or textures;  all it cares about is how far away an object is.  So it’s drawing its view of the scene with darker colors representing close objects, and lighter ones being further away.  It’s also worth mentioning that the light doesn’t turn with the player’s camera;  it’s always oriented with north at the top of the image.  In this image, the camera is facing approximately northwest.

In the light’s view you can (roughly) see the column I’m standing beside, as well as the trees that are casting the large shadows on the left.    (The more distant objects are not visible in the light’s rendering of the scene.  I’ll talk about that in a moment)

So after I’ve rendered the light’s view of the scene, I then draw the scene normally, but with a little extra logic running per pixel.  The logic goes like this:  For each pixel I draw on the screen, I figure out where that pixel is in the world, and then where that position would be in the light’s view.  If the pixel is further away than what the light could see during its render, then I know that that pixel is blocked from the light, and so I should draw that pixel as being in shadow.  If not, then it’s in the light.  And that’s pretty much all there is to it!

I’m a dirty liar

Actually, no, there’s a lot more to it than that.  See, the chief problem with shadow maps is resolution.  When the camera is close to shadows (as in this screenshot, and in almost all first-person perspective games), we really need a lot of pixels in our light’s view of the scene to keep our shadows from appearing blocky.  But we only need that high resolution for the shadows that are close to the view;  if we drew the light’s view of the scene at that sort of resolution everywhere, we’d have a terrible frame rate.

So what almost every modern first-person-view game does is to use a system called cascaded shadow maps.

A simple overhead-view of the player’s first-person view of a scene looks something like this.  The black trapezoid (“frustum”, in 3D) represents the part of the game world which the player will be able to see in a particular frame.  Anything outside of that frustum won’t be visible to the player, unless the player turns or moves.  The player is at the narrow end of the frustum, looking toward the wide end.

When viewed from the player’s position (that is, not from an overhead view like this one), an object at the narrow end of the frustum is drawn using a lot more pixels than that same object being drawn at the wide end of the frustum.  And this means that we need a lot more detail on things being rendered at the narrow end than at the wide end.  This is true for everything, and a lot of computer graphic technology is built around dealing with this “how can we draw high quality stuff close to the player, and lower quality stuff further away” problem.  For models, we have LODs;  simpler versions of models which can be drawn with fewer triangles, for drawing when models are far away from the player.  For textures, we have mipmaps, pre-filtered textures which are optimised for being displayed at smaller sizes on the screen.

Similarly, for shadows, we use cascaded shadow maps.  The idea is that rather than draw one big shadow map as we discussed above, we instead want to draw several, each covering a different part of the view frustum.  Traditionally, each shadow map (called a “cascade” in this context) has the same number of pixels in it;  often 512×512 or 1024×1024.  But one will cover a very small box in front of the player (thus having a very high pixel density), another will cover a larger area past that (thus having a lower pixel density), and there may be several more, each covering larger areas located further away from the player, and having coarser and coarser pixels.  In the screenshot here, I have roughly estimated the positions of three cascades to fit the frustum, and outlined them as grey boxes.

Under this approach, when rendering a pixel, we first check how far forward it is from the camera, and then based on that distance forward, we pick which cascade to look in to decide whether that pixel is in light or in shadow.  Right now, MMORPG Tycoon 2 has three cascades.  One covers the first ten meters in front of the camera, the next covers the next 50 meters, and the last covers the next 800 meters.  Each is rendering a 1024×1024 image from the light’s point of view.  I’m not certain that these numbers are the best choices right now;  It may well be that I’d be better off using more cascades with smaller images for each.  Or maybe the cascades should be spaced differently.  But fundamentally, the technology is in place and working;  it’s only fiddling with numbers that’s left to do to tweak the behaviour.

Almost every modern first-person game out there uses exactly this system for rendering their shadows, at least in outdoor scenes.  It’s easy to spot, once you know what to look for.  Because we’re rendering several different shadow maps at different resolutions, all you have to do is walk forward in a game, and look for a spot where shadows change their level of detail.  In an FPS, this will ordinarily appear as a horizontal line across the ground, where shadows visibly become more defined as you move forward.

So that’s cascades.  There are a few other issues to be considered, but they’re probably too far down into fiddly points to be of much interest to most folks.  I already talked about surface acne in a previous post.  The opposite of that problem is called “Peter Panning”, where you’ve biased the shadows so much that they actually detach from the objects that are casting them, causing the objects to appear to be floating above the ground.

Another common problem is shimmer at the edges of shadows, as the camera moves and rotates.  The trick to solving this is to move the light-view cameras in increments that match the distance between pixels in their shadow map, not rotating the light-view cameras to try to better fill the frustum, and not changing the sizes of the cascades from frame to frame.  Doing these things means that if I move the camera (say) three pixels to the right, it will generate exactly the same shadows as it did before, just three pixels away from where it was generating them before.  By comparison, if I moved the camera by a non-integral number of pixels, the shadows would still look very similar, but the precise pixels at the edges of a shadow shape would change, just based on rounding differently.  And this generates a very distracting shimmer around the edges of shadows.

That’s really all I have to say on the subject.  If I’ve forgotten something or if you’d like more detail on one bit or another, please ask in the comments!

Shadows, now working on both ATI and NVidia chipsets. Have not yet tested on a device using Intel integrated graphics.

Not sure how well these shadows will work in that situation.  But I’ll definitely want to release MS-2 soon, to hear about performance on more machines.  Hopefully in the next few weeks!

This isn’t the best shot of the shadows;  I’ll take another screenshot later tonight which will be a bit more dramatic.  But I was sort of startled to see this a moment ago:  AI developer cursors automatically casting shadows.  I probably should disable that, since they’re not supposed to literally be present inside the game world.  But it’s neat to see the shadows working.

It’s looking like I probably need a third shadow distance.  The close one is really needed for when you’re actually standing on the ground.  But for the shadows visible in the middle-distance here, they look a lot blockier than I’d really like.  So there’s still a bit more work to be done.  But overall, I’m pretty happy with where this is going.

In discussions about shadows, these sorts of odd striations and blemishes are generally referred to as “self-shadow surface acne”. They’re commonly caused by math imprecision when a surface (such as here, the mountain) throws a shadow onto itself.  The shadow-caster and the shadow-receiver are so close together (ie: directly on top of one another), that sometimes the math flips their order around.  This results in these interesting stripe patterns.

Hope it doesn’t take me too long to sort them out.  I’d really like to finish shadows today and get back to items.

The first requirement for this is to be able to render an orthographic projection.

Orthographic projections are actually pretty simple;  they’re a view which has no vanishing point;  objects don’t appear larger as they get closer to the camera, or smaller as they get further away.  Oddly enough, this is the traditional sort of view used for most “tycoon”-genre games.  The screenshot here is of a regular view into the MMORPG game world, taken from an orthographic camera.  And.. I think I kind of like it, once real shadows are in.  I’m starting to think about using this sort of view for general terrain editing and building placement modes, then going down into first-person perspectives for non-editing modes.

Until now, the VectorStorm library had no native support for orthographic views;  you could either render a flat 2D space, or you could render a perspective projection of a 3D space.  I’ve now added support for orthographic views to the vsCamera3D.  There are a whole host of changes under the hood which are needed to make use of this.  But the upside is that you can now ask a vsCamera3D for its projection matrix;  that’s no longer hidden inside the renderer.

To use an orthographic view, set a camera’s projection type to “vsCamera3D::PT_Orthographic”, and set its field of view to the vertical distance you want to be rendering.  The horizontal distance is automatically determined based upon the field of view and the aspect ratio (exactly as we were already doing on the vsCamera2D).

Tomorrow, I’ll be hooking up a couple of shadow map renders to this sort of view.  With any luck, I’ll get all the way through and be able to show demos of proper shadowing.

As I mentioned yesterday, there’s still a lot to do with shadows.  Most notably, right now shadows are only being generated for a single area of the game world (covering about half of the starting region).  Move away from there, and shadows vanish again.  I’m planning to implement cascaded shadow maps, which will result in shadows being calculated near the player, and with better resolution near the player;  blockier further away.  I’ll make an attempt at this on the coming weekend.

But what I really want to talk about today is the screenshot from yesterday, with all the red squares in the actionbars.  In a lot of ways, for me, those screenshots are more exciting than these ones with the progress on shadows.  See, what’s going on is that I’m in the middle of a massive refactoring of how action bars work.  And here’s why:

I’m adding an inventory.  Both for in-game characters, and for the player himself.  “Items” are now objects which can be carried around, and optionally perform a function when clicked upon.  And almost everything is now an infinite-use “Item”, including all of the terrain editing and building placement tools.  Even the button for stepping off of the graveship is an “Item” now.  Similarly, the various combat abilities of mmo classes are now implemented as infinite-use items, too.

Items are stored in various places (inventory, class abilities, etc), and shortcuts to these items can then be dragged into action bar button slots.  I haven’t yet implemented stacking and depletion of similar items, but that’s planned;  hopefully should have that working by the end of the week.  This support gives me the ability to (for example) give the player once-off consumable items, such as “construct a capital city”, without cluttering his interface with disabled buttons for all the time when he’s not allowed to do that.  It also opens the door to, for example, the player being able to de-rez a building, pick it up, carry it somewhere else, and then rebuild it elsewhere, at no charge.  I think I like that mechanic a lot more than the current “drag the building freely around the landscape” interface.

In a more obviously useful vein, it also allows the MMORPG game to have things like health potions, spell scrolls, and other one-use items, and have the AI players automatically understand how to use them.  These consumable items are all implemented in exactly the same way as class abilities, which means that they should slot straight into the AI action selection logic that already exists.

Originally, the idea for MT2 was to abstract away items, because of all the complexities that surround them (drop tables, loot frequencies, etc), but in the end, I really wanted to be able to give items to the player for the purpose of building the world.  I probably won’t expose the complexities of drop tables in the first release of MT2;  it’s not really what I want the game to be about.  I’ll just have AI game designers fill them out automatically.

So that’s what I’m up to at the moment!  Hoping to polish off the item interface this week, and get cascaded shadows over the weekend.  Wish me luck!

Some background:

VectorStorm’s graphic engine works based on a concept that I call display lists (OpenGL programmers, these are not the same thing as OpenGL’s display lists).  In VectorStorm, a display list is a set of rendering commands, converted into a binary blob of memory.  You can think of them as a small program that is run by the renderer.  Statements in these programs are things along the lines of “Activate the texture ‘grass.png’”, “Draw this list of triangles”, “Draw a line between these two positions”, etc.

Originally, VectorStorm’s drawing strategy was to create a display list, and pass it around to all the drawable objects in the world, letting each of them put their own drawing commands into that list.  In practice, though, switching from one type of material to another (e.g.: switching from drawing clouds to drawing grass) is much more complicated and time-consuming than almost anything else, and so I added a class which I call a Render Queue.  The Render Queue basically keeps a separate Display List for each material.   So when objects submit their “here’s how to draw me” instructions to the renderer, they’re added to a render queue for the materials that they’re using.  This means that, for example, we only need to set up the tree foliage material once, no matter how many trees are visible on the screen.  In the world of graphics technology, this is called batching — rearranging drawing commands to try to make them run faster.  This made a huge difference to rendering performance.  It also gives me a lot more control over specifying which objects are drawn in front of other objects, for things which don’t use the z-buffer.

For backwards-compatibility, the Render Queue also keeps a “Generic Display List”, which old code can use the same way that it used the original display list.  But that tends to be a lot slower, and has a lot more draw-ordering issues.

Almost every one of the “Testbeds” games, for example, still uses the Generic Display List for all their drawing.  But MMORPG Tycoon 2 is attempting to perform a lot more drawing and needs to perform more optimally, so almost everything in MT2 is now rendering using the batching system.  (In the screenshot above, only the MMORPG name, the bars beneath it, and the timing bars in the bottom left corner of the screenshot are still rendering via the generic display list — absolutely everything else is being rendered using the batching system)

Or at least, that’s what I thought.

This morning I discovered that absolutely everything in VectorStorm’s 2D rendering path was still using the Generic Display List, not the batched rendering system, even if the game was requesting the batched rendering system.  I had helpfully left myself a little comment that I should turn on the batched rendering for 2D, which was nice of me.  But there were some technical challenges in getting it up and running.  The largest of these was that I’m still sticking with VectorStorm’s weird coordinate system, which required some serious doctoring of matrix math to make it all work with OpenGL.

The long and the short of it, though, is that I now have 2D objects being correctly batched, the same as 3D objects.  This should give a sizeable speed boost for folks on slower GPU processors;  the 2D drawing in MMORPG2 has always been a much bigger drain on GPU performance than the 3D drawing, so anything that can help it out a bit will definitely help a lot.

I’ve got more stuff to talk about (particularly the reason for the red squares in the picture above), but I’ll leave that for tomorrow.  I’ve already babbled for far too long, tonight.

And maybe I’ll manage to do the cascaded shadow maps next weekend;  it’s too big a task to really attempt in my spare time during the week.

This is all very early stuff, and I’m sure that I’m going to want to completely rework the mechanisms by which this works.  But I’m pretty pleased at having things which are recognisably shadows in the game!  (And interacting nicely with the grass effect, etc!)

This isn’t ready for prime time yet.  It’s still very, very glitchy, and is far too computationally expensive for what it is right now (because I’m rendering into a 2048×2048 shadow texture).  And I’m not even certain that I really want shadows for the visual style of the game.  But it’s nice to finally have VectorStorm supporting them!

Also, MMORPG Tycoon 2 is now using my de-facto VectorStorm library development trunk, which is over at Github.  Which means that if you want to see the changes under the hood, they’re all publicly visible right now;  no need to wait for me to port them back from the MT2 codebase to the VectorStorm trunk.  (Note that there are no testbeds or samples demonstrating anything using this;  it’s just source code for the rendering architecture, and therefore probably not of much interest to anyone)

Notable new stuff includes:

• VectorStorm games can now implement custom frame drawing logic (defaults to old behaviour, if they don’t).
• VectorStorm now supports offscreen render targets, for television screens, shadow buffers, and other purposes.  Render targets support multisampling buffers and depth-only targets, in addition to normal texture targets.
• Games can now request individual scenes or groups of scenes be drawn into a special render target.
• Games can now override the normal lit/unlit/textured/untextured shaders with their own custom shaders.
• Game-provided vsShaders are now able to set uniform variables and otherwise set up their state, when they’re used.
• vsImage can now read back texture data from a vsTexture object (even one inside a render target).

I’ve just done timing tests from my laptop, here in Australia, sending this repository to freshly created bare repositories at my three big hosting options (timing via the standard unix ‘time’ command line tool):

Github:  58.588 seconds.

Gitorious:  76.964 seconds.

VectorStorm: 54.316 seconds.

This pretty much validates my subjective feeling that Gitorious felt slow to me from here.  (I suspect that these results would be different if measured from elsewhere in the world)

It does surprise me that VectorStorm outperformed Github, but it was only by about 10%;  not a big improvement.  And Github provides a lot of other features, so that’d probably be the most sensible place to put the VectorStorm library.

In terms of actually doing it, I’m still cleaning up the new library structure, particularly the cmake build structure.  But it’s getting pretty close to final now, I think.