The Utility of Head Mounted Displays

In my last post, I talked about various qualities of head mounted displays. However, today I want to talk about how head mounted displays could affect the future of gaming. In some ways, HMDs will probably change nothing about the user's game experience. Yet, they could be a useful tool for solving a few long-standing problems of game design.

The addition of a head mounted display would probably have a similar effect to increasing the graphical capabilities of a game console: increasing the visual appeal of a game. This would be particularly true if the headset was more immersive than a standard television or possessed solid 3-D capabilities, as I discussed in my last post. However, it is important to remember that this increase in visual appeal is limited in what it can do. For example, merely improving the graphics of a game can improve the reception it gets, but it won't solve any problems inherent in the game design. Fancy graphics or 3-D capabilities in a sense are novelties. They might draw in people, but they won't be enough on their own to sell a game, console, or peripheral device. In particular, decades of bad 3-D movies, the limited commercial success of IMAX, and the currently slow adoption of HD television demonstrate that technology that relies solely on an improved visual appeal is not necessarily capable of being marketable.

In summation, a high-quality, possibly 3-D, head mounted display will draw interest and attention, but will quickly be discarded as a useless gimmick if its creators and game designers rely on selling it entirely based on the improved visual appeal of compatible games. Thankfully for the head-mounted display, it does possess an unique advantage that can change game design in a somewhat significant way: what is displayed by the headset can only be seen by the person wearing it.

Having information that is limited to only one player can be a very useful game design tool when dealing with multiplayer situations. Back in late November, I discussed the problems that existed with local multiplayer (both co-op and vs.) on current console systems. Right now, on-line multiplayer is generally a superior experience to local multiplayer, because players don't have to deal with a split-screen when playing people over the internet. They get to have the entire screen to themselves. However, if multiple players each had head-mounted displays, that problem would go away. Everyone would be able to experience the full game experience. Furthermore, it would open up some interesting multiplayer design space, where one player could be using a headset, and another payer could be controlling the game in a different way on a TV screen.

I admit that this advantage is a relatively small one. Of course, head mounted displays can have a lot more interesting uses for video-games if they go beyond just a graphical display device and allow for unconventional control schemes. However, I will abstain from speculating about that impact of such a device until I get a chance to try one out. For now though, the utility of a HMD is mostly limited to an increase in visual appeal and an improvement in local multiplayer game design.

GDC Tech: 3-D Gaming Headsets

While I was at the Game Developer's Conference, I had the chance to try out two different Head Mounted Displays designed to create a 3-D gaming experience for the player. An immersive, 3-D headset has been part of the imagined future of entertainment for decades, and has appeared in near-future fiction such as the .Hack series for about as long. However, I don't think either of the devices I tried have actually been able to reach that level yet.

The biggest problem I found that was that neither headset was truly immersive. When wearing either one of the headsets, it felt like I was just looking at a really small television screen. The composite display still looked like a rectangular screen that easily fit inside my field of view. As a result, I still felt the same separation from the action of the game that I feel when playing on a regular television screen.

There is only one experience that I have found to generate a satisfactory feeling of being close to the action: an IMAX-Dome theater. A large-screen IMAX theater is so immersive that the visuals of a movie trick the viewer into thinking that the entire theater is moving. The reason for this is that the movie screen covers the spectator's entire field of view, including the viewer's peripheral vision, with a single unbroken image. A head mounted display should also be able to generate this kind of effect.

A problem with one of the displays I looked at was its lack of clarity. That system attempted to create a 3-D image by transforming the input feed from a game designed for normal 2-D screens into a pair of quickly alternating images that could be used by the glasses to create a 3-D image. They had a display of the modified input on display on a regular TV screen to demonstrate how it worked. However, this approach did not create a very clear image. Part of it had to be a goof on their part, since the text and UI of the game was not split into the two images. While it was readable on the TV display, it was constantly out-of-focus on the 3-D headset. However, the strobe-like alternation of images was discernible while wearing the headset. So, trying to make out any detail at all was difficult.

However, the 3-D display at the Intel booth did not do a particularly good job of creating a 3-D image at all. Perhaps it was just a poor choice of game, but I did not perceive much of a 30D effect while playing with their headset on. While I could read the UI and game text just fine, and there were no problems with clarity, the game just didn't feel like any different of an experience from watching it on a TV screen.

3-D can be a very difficult effect to achieve. However, I actually think immersion is not only easier to achieve than 3-D, but possible more rewarding. The feeling that the entire room around you is moving is a very strong effect that can really draw a player into a game. And it can simply be a matter of having a sufficiently high resolution headset that can cover the player's entire field of view. Trying to create a 3-D effect requires significantly greater engineering.

However, whether they are 3-D, immersive, or not, headsets could be a very interesting way of redefining the world of video-gaming. However, I will save that discussion for next time.

NeuroSky's New Gadget

Preparing for and attending the 2008 Game Developer Conference appears to have distracted my brother and me from posting here for a week. I guess it is time to get back on the regular posting schedule.

At the GDC, I had the opportunity to try out a gadget created by the NeuroSky company, a head-mounted device designed to read the brainwaves of a person who is playing a videogame, analyze those brainwaves to determine the player's mental state, and relay that information to the game being played. The gadget just needs to be placed on a persons head in order to read brainwaves (nothing difficult or obnoxious for the player) and seems to work quite well. It can't seem to interpret thoughts into direct game commands or anything like that, but it can evaluate the player's mental state and translate that into an effect on the game. The demo at the GDC demonstrated the device's ability to observe the player's attention and meditation levels, and the company representative I spoke to claimed that the system can also observe other things like drowsiness and anxiety levels. This technology is a far cry from the holy grail of being able to play a videogame using nothing but thoughts, but it is still an impressive technology that can be used to make interesting videogames.

However, despite the potential of the system, the demo at the GDC didn't do a very good job of showing how this technology could be used in a game effectively. The demo relied on the gimmick of "moving objects with your mind", even though the technology could not actually translate the desire to move something into actually moving something. Instead, they just had the overall attention levels and meditation levels serve as a "power source" for commands executed through mouse clicks. For me, "using my mind to move and burn objects" was just a process of clicking on an object, and then having my apparently natural high attention level (possible augmented by caffeine from my lunch) trigger the effect with no conscious effort on my part. That was not terribly entertaining. The other activity, lifting things with meditation, was just an exercise in demonstrating that it is not easy to control your mental state on command, and would not make any better a game experience than the other activities. Controlling action with mental state is not a viable way of implementing game controls, because it is far too imprecise and dependent on individual personality, That said, I do still think that this system can be used for games.

Rather than use the implementation seen in the demo, I think it would be much better to use a player's mental state to control the mental state of game characters. This technology can be used to synchronize the way the player reacts to situations in a game and the way the character controlled by the player reacts to those events. If the player pays attention, the character will pay attention. If the player becomes anxious and frightened, the character will become anxious and frightened. There are a number of interesting specific implementations I can imagine for this:

1) Personalizing animations in an MMO. A player can set specific animations to play when his mental state changes. For example, the player might have it so that his character yawns or stretches when the player is drowsy, or jumps up and down when the player is excited. In fact, this cna be extended so that characters can have very different animations based on the mindset of the player, making each character act like the character itself is feeling the emotion and expressing it through natural body language.

2) Implementing some kind of action game mechanic in which the mental state of the player enables different kinds of styles and techniques for the character. If the player is highly focused and paying attention to a single target, then the character can use powerful moves to fight single foes. If the player has achieved some kind of zen-like calm while fighting countless enemies, or perhaps is bored and wants to just get past a part of the game, then the character might reflect that by being able to use moves that defeat large numbers of weak enemies quickly.

3) Altering the atmosphere and mood of a landscape. Make it so the world can reflect the mental state of the person playing the game, If the player is happy, the game world will be bright and colorful. If the player is afraid, the game world will be dark and terrifying. Or perhaps the environment can be made to oppose the mental state of the player, and try to cheer up a depressed player or make a happy player anxious.

There are many possibilities, and I am very curious how this kind of technology will be used by the industry in the future. It might be a while before brainwave-reading technology is packaged in with every game console, but I am sure that some form of this technology will add a lot to videogames in the future.