Monday, December 8, 2014

Gear VR now on sale + some good articles

Looks like the Samsung Gear VR has just gone on sale!

Unboxing by Road to VR:

Q&A with Samsung VP Nick DiCarlo:

Tom's Hardware: "Samsung Gear VR Suddenly A Real Thing, Yours For $200",28178.html

Road to VR's Gear VR Review Part 1:


It will be very interesting to find out what Samsung & Oculus learn from this product, and how that translates into improvements for the next one (and for the Oculus Rift).

Thursday, December 4, 2014

Titans of Space (Gear VR) - Recent Optimizations

I mentioned in my previous post that I removed sterescopic 3D in favor of player comfort and a few bells and whistles.   I then updated the post stating that I was able to provide an option to enable a 3D cockpit.  This post goes into some detail.

When I first started out with Titans of Space, it was using a layered camera system to render the cockpit on top of everything else outside of it.  This made things easy re: depth precision, and keeping the player at the origin.   Then, when working on the Gear VR port, I discovered that there were some performance issues with using multiple cameras and received advice to take a different approach.   The result was a single camera rig and carefully fitting the scale of everything into just that one view, and taking care of floating origin, etc.

As is, enabling full sterescopic 3D in Titans of Space for Gear VR results in less than 60 FPS and thus unacceptable judder for large objects that are moving across the player's view (such as planets). Note that for Gear VR apps where the player remains stationary and there isn't a lot of movement through the scene, async timewarp comes to the rescue and it will remain completely playable at less than 60 FPS with smooth headtracking.

Now that some time has passed with the Oculus Mobile SDK maturing enough to see a public release, and with newer & faster generations of hardware (the Note 4), I took another stab at a layered camera approach.   The idea is that if you can render the distant scenery just once and display it to both eyes, with the nearby scenery rendered twice for stereoscopy, you can avoid rendering all that distant scenery twice.   This has already been effectively implemented in a handful of demos for the DK2 (Time Rifters comes to mind).

The key question is whether it's worth it to do so, as there is overhead in rendering distant and nearby scenery separately, even if the distant scenery is only rendered once.  Turns out, this 2D/3D mode put Titans right on the edge of 60 FPS.   It was good enough most of the time that it was worth including as an option, but not perfect enough to make it the default mode.   By default, it will just render everything for the left eye and re-use it for the right.   While in 2D/3D mode, I also turn off chromatic aberration correction as that has about a 1ms GPU cost.

Another thing I optimized is Earth's appearance.  It's the first thing the player sees so it has to make a good impression.  The DK2 version renders Earth using three spherical meshes, with a high number of polygons in each.   One for the surface, one for the atmosphere, and one for the clouds which float above the surface.   This makes vertex count soar and incurs a ton of overdraw on the Gear VR.  So I simply got rid of the floating clouds, and they now appear on the surface texture itself.   The normal map for the surface has also been flattened out wherever the clouds are so that the clouds don't look like they are painted on.   Anyway, the end result looks good and doesn't kill performance (as much as it used to), and it means the clouds cannot move independently of the surface, but that's a small price to pay.

What's next for optimization:

I noticed in one of the Samsung Developer Conference (SDC) presentations that checking the box for GPU Skinning actually provides a benefit even if you don't use skinned mesh renderers, because it "activates the DirectX11 profile".  That was not something I'm familiar with so I'll be looking into that.  I currently have that unchecked simply because I didn't want Unity to bother worrying about a feature that I'm not using.  Very curious if this provides any kind of a benefit!

There is also the matter of texture compression.  When setting up texture import options (either on a per texture basis, or in the override in Build Settings), there are several highly efficient texture formats to choose from, all with tradeoffs.   "ETC2 (GLES 3.0)" is the one recommended in the SDC presentation, and yet every time I fiddled with that in the past, I either get unacceptable texture quality, or there isn't any perceivable benefit to performance.   However, note that it says GLES 3.0 in parentheses.  I'm running with the "Force GLES 2.0" option, as forcing 3.0 causes an intermittent crash that Unity is aware of and is actively troubleshooting.   It's my understanding that getting the app to work in GLES 3.0 mode may also provide a large performance boost.   In any case, this is another example of why it helps to build something from the ground up for Gear VR -- I'm starting with an experience where texture quality is a big focus, and would rather not take a step backward there on a platform that is known for having very high resolution.

Speaking of resolution, the text in Titans of Space is just as legible on the Gear VR as it is on the DK2.   Despite the higher resolution display in the Gear VR, it is not much more legible, and the real benefit you will see in Titans of Space for Gear VR right now is a reduced screen door effect.   The reason for this is that I am still using eye textures that have a lower resolution (1024 x 1024).   I have experimented with increasing the resolution of the eye textures but wasn't able to raise it enough to see an appreciable benefit to text legibility while still remaining above 60 FPS.   As a test, I made an empty project with some spheres and cubes with detailed textures painted on, and ramped up the eye texture size to 2K x 2K.   The difference is absolutely unbelievable!  My mind was blown, and this is another big reason I am looking forward to continuing my optimization push.

The eye textures remain at 1K x 1K, but I also experimented with rendering only the information panel text at high resolution with everything else being the standard lower resolution.   It was a very successful test, but the overhead for it is unfortunately too high for there to be any benefit at the moment.  Perhaps other optimizations will enable this to be a possibility in the near future.

Finally, the UI system that I use in Titans of Space is not very drawcall-friendly.  It takes a drawcall per label, although many of them are being dynamically batched.   The existing UI code makes it easy to support multiple languages across multiple fonts, with smooth outlines around each character (unlike NGUI).  However, it uses bitmap fonts, and requires a bit of preprocessing work to get the right look for each font.  It also takes up a lot of texture space and bloats the final app size.   When you open up the options or help HUD, framerate clearly suffers and is a simple demonstration of how brutal it can be to add a few more draw calls on a mobile device.   I aim to overhaul all the UI stuff as soon as I can find a good replacement that can address each of these weak points.   Unity 4.6 with its new UI system is now officially out, and as far as I am aware works just fine with the Mobile SDK, but am not yet sure if it will meet all my needs.

Optimizing "TNG Engineering" for the DK2

After releasing TNG Engineering for the DK2 (which is now up on Oculus Share!), I received some requests to make a post about optimizing this to run so smoothly for so many.

To be quite honest, this experience isn't anywhere near optimized as it could be, but I did place a focus on avoiding hiccups / framerate hitches / frame freezes and generally getting the FPS up as high as possible with a minimum amount of work involved.    Just as an FYI, I used Unity Pro 4.5.5p3, and some of this may no longer apply in Unity 5.

Don't Move:

TNG Engineering is your standard static scene, where nothing moves. That alone opens the door to static batching, which is a big part of getting that "greased lightning" feel.  In my demos I tend to have a single Unity scene that fades in and out of different areas, such as the "face forward and press space" prompt, the title screen, and the main environment itself - in this case, the Engineering Bay. Only one area is active in the scene hierarchy at a time and the camera is moved from one to the next. So, in order for static batching to actually work, geometry marked static has to be active in the scene hierarchy when Unity is generating a build.  It cannot be inactive at startup and then later activated at runtime and still be able to participate in static batching. That little caveat was tripping me up for a while, but now I've got a script to work around that, with great results.

Also, since dynamic batching isn't providing any benefit (either to draw calls or to the framerate) for this scene, I disable dynamic batching just so that Unity doesn't bother trying.

Curing Hiccups:

Fading to black is a pretty straightforward way to cover up any heavy-lifting being done to load things up.  The key is to make sure that everything is loaded up right then and there, and not leave anything out that will cause a hiccup later.  Larger / more complex programs may instead need to carefully manage memory usage and resource loading in an incremental fashion, but this post is about simple scenes.

The player first starts out in a short hallway leading to the Engineering Bay, so when the player rounds the corner to see the rest of it, there can be some pretty serious frame freezes as things are being loaded and rendered for the first time. There are seemingly a lot of tools that Unity provides to address this, such as preloading all the texture resources, warming up all the shaders, uploading meshes to the GPU, etc, but the approach I took in TNG Engineering to make sure everything is loaded into memory and onto the GPU is to simply render it normally.

So in TNG Engineering, while the screen is faded out, the camera will actually move to 3 different positions and orientations and render one frame each, before ending up at the hallway where the player starts. This gets all the heavy loading out of the way while the player can't see anything. This approach may not work for every type of program.  For Titans of Space, instead of moving the player around, the program instead renders every planet and moon ahead of time in front of the player just before the tour starts, and then hides almost all of them and puts them back where they belong.


Direct lighting has a pretty significant impact on performance, and realtime shadows even moreso.  In TNG Engineering, no realtime lights are used (and thus no real-time shadows).  Instead, all lighting and shadows are baked beforehand.  This can increase the size of your project with lightmap textures, but it's a non-issue next to the boost in performance and visual quality.

Collision Detection for Player Movement:

Since I am letting the player move around, much of the geometry has colliders to prevent movement through the walls. In cases involving detailed geometry, it is usually overkill to create a mesh collider for exact collisions, so these are instead loosely represented by primitive shapes like cubes. There are still plenty of mesh colliders in use, though.

Collision Detection for Positional Head-Tracking:

And finally, since the DK2 has positional head-tracking, I like to fade the screen out to dark blue as a function of distance from the player's head to a wall or surface. My approach to this is to use the CheckSphere physics function once a frame, with a radius that is equal to where fading out just barely starts. If that returns true, then I do up to 2 or 3 more CheckSphere calls to narrow down how far away the player's head is from a surface. I end up with a granularity of 8 steps to fade out completely, which looks smooth enough in practice, and yet most of the time there is only going to be the one CheckSphere call to keep things performing well.

That's about it!