Porting OpenMW to Ogre 2.0

In the past few days I’ve been taking a closer look at Ogre 2.0, to evaluate if and when we can start using it. I’m now confident that a prototype is feasible. Some work has been done on the prerequisites and an outline has been created for problematic issues. I’ve also listed some new features in Ogre 2.0, besides performance improvements, that are particularly exciting for us.

Dependencies

The first step is to port MyGUI. Luckily its use of Ogre is fairly minimal and abstracted via a Platform interface. It didn’t take long to get most demos working and I arrived at the following patch. One interesting change is the rendering callback. Previously, MyGUI registered a RenderQueueListener with Ogre’s scene manager to be notified when the “Overlay” render queue is hit. This no longer works in Ogre 2.0, because empty render queues are skipped. Curiously, I found the following comment in the Ogre source code (in 1.x versions, too!):

// NB only queues which have been created are rendered, no time is wasted
// parsing through non-existent queues (even though there are 10 available)

My only explanation for how MyGUI rendering could have actually worked in the first place is a bug in Ogre 1.x, causing empty queues to be traversed anyway – contrary to what the comment says.
After changing MyGUI to render from the frameRenderingQueued callback, it works. This is not a clean solution, and I’d like to leverage the compositor system instead – but custom compositor passes are not supported in Ogre 2.0 yet (though planned).
Also, MyGUI’s Render-to-texture and the RenderBox demo have not been ported yet. This is a tricky part, because direct updating of render textures no longer works as it used to: all rendering has to go through the compositor system.
A couple of loose ends and I’ll probably have to revisit them later, but good enough for use in a prototype.

On to problematic points in OpenMW itself…

Tag points

Tag points are used to attach an object to a bone on a character’s skeleton – say, a weapon on their hand. This feature is widely used in our character system (not just for equipment, but also for supporting Morrowind’s segmented body parts).
Unfortunately, it hasn’t been ported to Ogre 2.0 yet:

TagPoints isn’t going to be fixed soon. First we need to integrate the new skeleton system into Entities

I’m considering to work around this for now by creating tag points as regular scene nodes and updating their transform manually every frame based on the bone transforms. This is likely not very efficient, but at least it works.

Scene node names

Currently, we use Ogre’s auto-generated scene node names to identify objects in a few places (whose idea was that?). This will not work in Ogre 2.0, because scene nodes don’t have unique names anymore.
I don’t expect too much trouble here – a search for getHandle() returns 93 hits and most of them are actually just getHandle() == “player” (*grumble*)
So, how do we deal with this? I’d like to remove the getHandle() function. We don’t really need it. The only legitimate use at the moment is for connecting Bullet’s rigid bodies with their MW-object for ray query purposes. We can easily change this to storing the object’s MWWorld::Ptr instead of the scene node name. And good riddance to World::searchPtrViaHandle, because it’s terribly inefficient.
If that plan fails, we could just use the unique scene node IDs (uint32) offered by Ogre 2.0.

Future

And that should be everything we need to worry about for the initial port. But there’s much more exciting stuff up ahead:

New material system

In 2012, I created a library for Ogre called shiny. This is what we currently use to effectively handle shader permutations, among a few other things.
My plan was to contribute a similar system back to Ogre directly, but as it turns out dark_sylinc has beat me to it. He’s developed a new material system for Ogre 2.0 in feature parity with my proposal. Some of the terminology has changed, but otherwise the proposal gives you a good overview of the new system.
It’s currently sitting in a feature branch and not ready for prime time yet. It will also require some porting on our end – notably, assembling materials is now handled by the user in a callback, rather than handled generically by shiny’s material template system. This is a welcome change for me, as I always felt the template system to be somewhat limited and unsatisfying. It wasn’t used much in OpenMW anyways, since we have to convert materials on-the-fly from the NIF format.

While all this is still in development, it’s good to know that we can continue using shiny in the meantime – it builds and works just fine with the current Ogre v2-0 branch.

Hardware requirements

So far, OpenMW requirements have been fairly low. Shaders are used by default, but not strictly required.
With the current Ogre v2-0 branch, this hasn’t changed yet, but the fixed function pipeline has already been removed in a feature branch, along with the DX9 and GL2 render systems. So eventually the minimum requirement will be GL3 compatible hardware.
If you think you might be affected, don’t worry – we can keep support for compiling with Ogre 1.x in a branch.

Another question is whether we can keep maintaining DirectX support on our end. Ogre’s API does its best to work rendersystem-agnostic, but unfortunately writing shaders doesn’t work that way. Currently, we use a porting header allowing us to generate shaders for both OpenGL GLSL and DirectX HLSL syntax from the same code. I don’t know if this will still work with the new material system. Shiny used the generic, but slow boost::wave preprocessor, the new material system uses a custom one. There’s a chance the header will trigger obscure edge cases or unimplemented features in the preprocessor. If that’s the case, then we will write shaders directly in GLSL, and DirectX support will be thrown out. I have no intention to maintain two near-identical copies of all shaders.

Animating culled objects

In Ogre 1.x, culled (as in: not visible in the view frustum) objects do not have their animation and skeleton updated. Sounds sensible at first, but this can cause problems when the animation drives collision objects or other gameplay elements.
It’s funny that we’ve been hit by that exact issue after Matias called it:

Frustum culling at fine granularity conflicts with DOD (Data Oriented Design) paradigms, and thus reduces the overall performance when looking at most of the skeletons (which is most likely the general case). Furthermore fine frustum culling is useless in modern real world applications where there are like +4 camera passes (reflections, shadow mapping) as eventually everything ends up getting caught by the camera. Also some games need to get the skeleton data for collisions or logic, despite not being on camera

In Ogre 2.0, skeletons are always updated, even when not in the view frustum, so the above issue and similar ones should go away.

Coordinate precision

In cells far from the coordinate origin, objects begin to shake violently. This is due to precision issues when transforming individual vertices with large coordinates.
A common solution is to treat the camera as the world origin for rendering purposes. An option for this exists in Ogre 1.x, but I haven’t been able to use it due to minor bugs. Camera-relative rendering has been redesigned in Ogre 2.0. This gives us a perfect excuse to revisit the problem and hopefully get rid of it for good.

Non-uniform character scaling

A missing feature in OpenMW is scaling the width of NPCs by their “Weight” property, so most NPCs currently look a bit too skinny. The reason we can not currently support this is down to Ogre: when non-uniform scaling is used on a skeleton root, the scale is treated in bone-local space. This does not match our needs; rotated bones (e.g. arms) become longer instead of wider.
It’s good to know that this issue has been taken into consideration and fixed in Ogre 2.0’s new skeleton system, although I’m not sure when we can start using it (it’s still labelled as experimental and disabled by default).

That’s it for today. The next post will be when I have substantial progress to show.

5 thoughts on “Porting OpenMW to Ogre 2.0

  1. Matias N. Goldberg

    Interesting read. It’s always nice to hear some feedback!

    “My only explanation for how MyGUI rendering could have actually worked in the first place is a bug in Ogre 1.x, causing empty queues to be traversed anyway”

    Yep! The old RenderQueue had contradictions like this, which was causing performance bugs and other weird behaviors.

    “In Ogre 2.0, skeletons are always updated, even when not in the view frustum, so the above issue and similar ones should go away.”

    A couple days ago I noticed that Entity’s legacy skeleton animation system still is updated based on camera visibility (but not for instanced entities apparently). This can be revisited though (particularly due to consistency).

    “Also, MyGUI’s Render-to-texture and the RenderBox demo have not been ported yet. This is a tricky part, because direct updating of render textures no longer works as it used to: all rendering has to go through the compositor system.”
    It’s not as tricky as it sounds. I’ve found that often you’d create an RTT and with it, a viewport. Thus you end up saving two pointers.
    Forget about the viewport (you won’t be creating it) and create a workspace instead just for that RTT (whether you share an existing workspace definition or create another one is up to you and depends on the job).
    Now, instead of calling renderTarget->update(), you’ll be calling workspace->update() (unless you decide to let it be automatically updated).
    That often does the trick. Look at the CubeMapping demo to see how the RTT is manually set to work with a workspace.

    Cheers
    Matias

    Reply
  2. DaVince

    Keeping a dev blog for OpenMW around like this is mighty interesting to read. Added to my feed reader! 🙂 It’s awesome to read that performance increases are coming with this porting process.

    Reply
  3. Pingback: It’s Ogre | OpenMW, Open source Elderscrolls III: Morrowind reimplementation, Morrowind remake, Morrowind remade,

  4. Pingback: To Ogr(e)! | OpenMW, Open source Elderscrolls III: Morrowind reimplementation, Morrowind remake, Morrowind remade,

  5. Pingback: » OpenMW: Transitioning From Ogre3D to OpenSceneGraphThe Ultima Codex

Leave a Reply

Your email address will not be published. Required fields are marked *