Augmenting The Elder Scrolls Online : forget presence embrace NirVRana:VR+3D HMD gaming on Oculus Rift, Cinemizer OLED+ smd ST1080 HMDs+Why TESO is going to be THE VR/HMD game of the decade

Whilst the real world shouts of the arrival of a virtual one, we are yet to find the perfect game or genre to play in such a VR or virtual world, with much controlled and one sided debate(?), some of it in shorts, on what makes a good VR or HMD gaming experience ..usually by the developers of said VR hardware or investors in that hardware ...which now over 3 years hasn't produced a single killer VR game.. or head set.. 

Finding that game ..does not require redefining game design, engineering or game development skills 

JC thought it was Doom back in the pre dev kit one days, the community screamed for Sword Art Online, still does, space combat and puzzle games dominate the current indie VR scene, that Sony see the limitless possibilities of VR as  demonstrable by being locked in a shark cage really speaks volumes....all..unsurprisingly fall short of young Johnny or Jenny Minecraft's expectations ...much as "VR" and HMD hardware does ..at this embryonic stage.

None of these  are THE game that will be played most by HMD users,  or ever I venture have the honour of being requested by Game of Thrones cast members .. Hodor!!

Finding that game requires a VR gamer .. And great presence of mind (pun intended Mr Abrash)  .. 

The Elder Scrolls Online is THAT game *

Q.E.D.

* outside Minecraft for ages 3 and 1/2 to 7 and 3/4.. 

 let me show you why ... got HMDs? Then light 'em if you gottem..

It was a coastal town... they were about to throw down..

...  dancing singing launching fireballs into the air with horses a leaping and a few people and non people attempting to do unsavoury things on the floor .. unsuccessfully I might add .. 

This was the scene 27 hours( so Raptr tells me)  into my 2nd Beta test of The Elder Scrolls Online..  the end of Beta... celebrations.. celebrations that I, as Abrash would say was able to grace with my "presence" 

Every day is NOT silent and grey ..

27 hours plus however much time I spent in the first Beta in Tamriel in TriDef 3D using three different HMDs the Cinemizer OLED, smd ST1080 and Oculus Rift Dev Kit (1) .. attaining a state as close to NirVRana as current tech allows  

NirVRana : Defn : a Zen like state of being where every one of your senses is functional and experienced in a game or VR world indistinguishably from the same sense in reality :Zen VR masters are able to transcend reality (and kickstarter) altogether and exist on another level ..some have  mistaken for the Hell of Doom 3 ..or more recently a shark cage on PS4..  NB "presence" is but a grain of sand soiling the long and winding path to true NirVRana .. NirVRana also differs from presence in being a deliberately comical construct ..

It was on the path to NirVRana .. lost in Tamriel..or perhaps just high from Rift Nausea and foam.. the following startling truths were revealed to me..

1. The Elder Scrolls Online is the finest HMD or potential  VR experience in existence..stuff all your indie games or so called VR flight or combat sim, stuck in a shark cage demos ... Tamriel was made for  3D and VR which was made for Tamriel .. this is so cool it may endanger the arrival of the Facebook Metaverse ....as Team Oculus give up on dev chores and go live in Tamriel.. perhaps Abrash, JC and Palmer will meet in Tamriel and craft Dev kit 3.. safe from the Reddit hordes.. and send out an enchantment to 3D Print it in the real world .. wait that's MR .. let's not get ahead of ourselves ..

2. The Elder Scrolls Online is going to be THE most played and requested game on HMDs of all kinds in LFOV 3D, WFOV Facebook Oculus VR and Sony PMVR ( Project Morpheus VR.. new tech ...new acronyms)

3. If you are not playing in 3D  ..you are not playing... the game launch screen alone with it's brook extending off into the distance ..will floor you .. 

1080p side by side TriDef DX11 3D view on Rift, smd ST1080  and Sony Project Morpheus HMDs (yes you can.. just view this through a browser on PS4) I dare you (JC) not to say wow when you experience this ..in motion

1080p top bottom  TriDef 3D view on Cinemizer OLED, smd ST1080 and Sony HMZT series ..you may notice saving up for all these HMDs ..leaves me no money for footwear.. wait..  long haired hippy type...barefoot .. I can hear the venture capitalists knocking at my cellar door..

4. If you are looking for a game to sell your HMD ..or VR .. use TESO .. 

5. If you are Bethesda .. reading this .. very little work needs to be done to optimise TESO for VR and non VR HMD use .. in fact none for LFOV HMDs.. 

Walk with me then in VR Tamriel .. as we attempt to close the Rift  between  great gaming, great VR and HMD gaming ..and VRy bad Rift puns and jokes..

Taking a look over the course of the last two TESO Betas .. how and why the Daedric Lords revealed this prophecy to me .. 

Many VR bridges to cross .. 

The Elder Scrolls Online bridges the gap between legacy gaming and VR gaming .. better than any other game  to date..

And crossing that "bridge" in Tamriel, is a stunning experience wearing your ST1080, Oculus Rift .. or Cinemizer OLED HMD .. time for some VR visual metaphors... 

the bridge between gaming and VR gaming

is one I crossed many times in The Elder Scrolls Online Beta 

sometimes having to battle the odd Rift Troll blocking the way ... something I am at least a Level 50 veteran at already..

A tale of Two Betas : 

I played the opening level of TESO using the Cinemizer OLED HMD with Zeiss head tracking in 1080p TriDef 3D. With this LFOV micro OLED HMD you feel the dark, 0 nausea and 0 visible pixels..the head tracking just works... On a 40 inch display.. you wear put on and take off like a pair of shades .. 

This is the HMD that you take on a plane with you, or can use in Shibuya Capsule hotel .. all without any discomfort.. this is one of two HMDs ..that I had with me at E3 and Eurogamer last year ( both together weighing less than a Rift) and could have shown the devs of any game on any device ...including TESO.. you as a developer have to do nothing to alter your code for .. your game already works .. on PC, console, next gen Console, iOS and Android phone... What no one told you? Naughty Internet.. 

try this scene with your HMZT , smd or Cinemizer to FEEL the depth of the environment ..

The opening level and vast tracts of ESO  are entirely set in darkness, darkness you feel using the Cinemizer OLED .. It turns what is a basic grind in a dark environment that your normal TVs and monitors struggle to place you in ..into a real experience .. thousands will have ground their way post haste through this stage to get to daylight asap .. with their PC monitors lighting up the darkness a grey back lit background .. with flat characters blending into it .. you know that playing PC games on a monitor in daylight experience .. On the Cinemizer you feel the 3D and the darkness behaves like darkness should .. which is why you will see me looking round with head tracking .. stand still and admiring how micro OLED 3D augments the already stunning game world.. even the character creation screen literally stands out ..

It helps that ESO's  designers .. design, craft and enchant their gaming world with as much fervor and dedication as it's most loyal players craft and enchant armor, weapons and spells.  

The result  is the most beautiful, complete and believable gaming world to date.. and I am not talking polygons, tessellation or FPS rates here .. these guys are craftsmen first, artists second and engineers last... the basic design ethos of TESO is to make you believe and sense you are in another world .. as close as it gets with today's tech.. This true of all Elder Scrolls games going way back to times pre dating GPUS. 

With the Cinemizer and other HMDs  you start to go into tourism mode ... of course you know you are not there .. but you might as well take a look around whilst you are...

a bland boring shot right ...wrong in 3D on the Cinemizer those chains are high up extending out towards you and you just looked up to notice them via head tracking ..the room is empty because you are the only one that noticed .. this detail or experienced it ..every one else just ran through a flat corridor on their screen .. missing the vertical detail ..the detail that some designer added to make you feel you were present in a game world rather than corridor.. 

on a PC monitor this is  a bland lifeless normal picture ..played with the Cinemizer OLED you should FEEL that hand reaching out of the darkness towards you ..simple augmented reactions like this contribute to overall immersion 

Why is any of this relevant ? 

My Dark Passenger ...

In TESO, atmosphere, layers of  darkness punctuated with neon magics, glimmering fire and shimmering water with trademark Bethesda sound scapes all combine to place you in the game world better than any other game  ..to connect you to it. That's before we get to the story or characters . Thus I can totally subjectively ( that one's for you JC) say Mr Abrash that TESO is a beyond presence experience .. as close to NirVRana as current HMD gaming gets 

NB HMD use and VR are a subjective experience..much like your own individual experience or memory of a book, film or song...the challenge is how many people you can bring along for the ride, share that experience..and maintain that experience .. over the course of a full game ...in the case of TESO ..this has to work over months to years .. 4 minutes of "presence" in an empty room ..is just a beginning..

These are some shots I took as I traversed this level, noting elements of  the game world that stand out and are augmented by HMD use.. 

Fire

in the game this is a fleeting moment , incidental detail , which in 3D on a HMD with head tracking just makes you want to stick around and  "experience" the game world..plus it looks stunning ..

Light and Dark

less is most definitely more TESOs most atmospheric locations in 3D on HMDs often appear to have the least physical  detail but generate a deeper "presence" than high poly counts or complex level design... 

Depth 

as you can see I spent a considerable amount of time trying to capture and attain the beyond "presence" state of NirVRana ..

Yet More Depth

which without the BS means TESO in 3D on a HMD will simply floor you .. (if the Rift nausea doesn't ..though Mr Zuckerberg..for health reasons please remember to take breaks when using Dev Kit 3 ..)


Scale 

the Dude Huge of this Tamriel prison is actually the size of a mouse in real life though his VR alter ego would like to disagree with you..and anyone else critiquing his friends..looks stunning on the HMD though..

Elevation 



Height 

whether by divine providence or happy accident of design TESO is the perfect HMD and VR game ..the closest you will get to attaining NirVRana  with this generation of gaming ..or HMDs 

I purposely included shots with darkness present as : Day night cycles and darkness are used as game mechanics in TESO .. the world looks beautiful in 2D in daylight scenes ..but generates some of it's finest moments and derives a lot of it's atmosphere from dark catacombs, night time battles .. the haze of dusk ...

The micro OLEDs of the Cinemizer really augment this experience .. which is why for me I looked forward to every dark dank dungeon and day night transformation in ESO .. after experiencing the intro level this way ..

You can play for any reasonable length of time that you would on your PC monitor.. I only stopped to move on to other HMDs .. 

This is the above level being played on the Cinemizer OLED : Watch my Cinemizer ghost to see the head tracking at work.. over a sustained period there is no perceptible drift, lag or random failure present as you can see JC ..

Issues 

HUD text needs to be larger, other than that TESO is perfectly suited to play with the Cinemizer OLED HMD, right now .. just plug and play using TriDef's standard display top bottom, frame packed 3D .. and at 1080p (scaled) res too.. 

I made it through the wilderness.. 

And went back into TESO using the siliconmicrodisplay ST1080 HMD , now with side by side TriDef 3D and true native 1080p res.  With a screen size that approximates to a 100 inch display and weight around the same as the Cinemizer .. 180g .. with the Zeiss head tracker attached.. 

Elevation 

Depth

height 

feel any of that?

With this HMD  we can use top bottom, over under and frame packed 3D .. Depending on the game you play ..the way it is designed ...one method may be more effective or suited to the way your brain interprets 3D .. I chose 1080p side by side for this test : partly because you cannot take snapshots of frame packed 3D...

I use one of my ST1080s in transparent or AR mode always .. which is not ideal for so called immersion but let's me type this on the monitor behind me, through the HMD screen on which I am playing TESO in 3D ...with head tracking.. Interestingly as long as your background is not a powered source of light or sky you still see a near opaque HMD screen.. even with the light blocker off .. 

Issues : none  the 100 inch TriDef 3D screen looks stunning, that I am powering it with a USB lead never gets old . The ST1080 is the first HMD you can read that text clearly on that 100 inch display ..ideally though we want to move the HUD to the centre and enlarge text in menus.. 

Both these HMDs are portable, I took both on the same flight and gave one to my fellow passenger whilst I used the other ... 

If your HMD is not portable you  need to be near your PS4 or PC to feel that ole black VR magic  Abrash coined as "presence" and that I am bashing here ... 

you are going to want to stop and watch the sunlight glimmer on the water that ripples and bends your shadow that you can feel the distance to below you .. on the road to NirVRana 

This is an example where versatility of function of light weight LFOV  HMDS trumps a lack of the same in WFOV VR HMDS of this generation...

You feel 0 nausea or see 0 screen door or pixels using either.. 

Isn't it Ironic ..

That I then went and spent the entire second day of the TESO Beta playing on the Oculus Rift

The Elder Scrolls Online Beta 2 1080p SBS TriDef 3D Oculus Rift Dev Kit 1

Method  : this uses my tried and tested, non patented method of sans Warp Rift gaming

or How to play a DX11 game using TriDef 3D on the Rift 

Do not use the Oculus Rift Beta settings in the TriDef menu!

1.Set your Display to 1080p res or 4K if you are using Dev Kit 3 (Marshmallow Unicorn)

 2.Using TriDef 3D set the display to standard, 3D model to  side by side  

2.1 Start TriDef 3D and add a game manually , adding TESO 

3. Select a generic profile for TESO 

4. Start the game

Now at a very basic level the Rift ( or Morpheus) will display side by side content as any 3D TV would to your eyes ...in you guessed it 3D.. 

The level of warp, geometrical warp that is done in software seems to be more significant than the level of optical warping the Rift lens does ..at least in Dev Kit 1 .. what this means is even without the geometrical warp the image you see is not bent out of shape ..as some people expect. 

You do lose the outer 4 corners and base line of the screen..but you still get to play TESO on your Rift in DX11 ..whilst we wait for the update to the Rift Beta .. and in a game like TESO you turn to turn the FOV anyway ..  

Now hit Alt Shit V to switch off Power 3D ( you want it on for the Cinemizer OLED and ST1080 and off for the Rift). This should deal with some cross talk or doubling of the fore ground .. with a third person game you may need to adjust TriDef settings to prevent the third person model doubling ..there are several methods of doing this .. using screen depth and custom focus .. mess around with these and you will find a setting that works for your particular game and your individual anatomy .. the short cuts can be brought up on the PC monitor you are mirroring the Rift content to by hitting ALT SHIFT and F1 together..

Once you have found a setting save the profile, you can apply this to other games you want to play with the Rift. You can play ANY PC game in this way.. this is how I was TVRKING Arma III and Bioshock Infinite back in May 2013 ..  

ISSUES : most of the HUD is useless, but we can move the HUD or play without it .. I could play the whole game this way .. after all why would there be HUDS in VR .. surely this would diminish the sense of "presence" and certainly lead you to stray from the path of attaining NirVRana .. 


there is another issue that may diminish the Abrash Presence or thegameveda NirVRana level you attain using the Rift :

Nausea : yes you will feel some , but playing sans Rift warp has always helped me with this .. I played for hours .. as you can see .. what was also critical is the head tracker I use does not fail or alter function randomly.. 

And here we come to the reality of wide field of view HMDS at this stage in HMD evolution all WFOV HMDs will produce nausea due to the fact that honestly WFOV HMD design is at the Stone Age part of it's evolution .. place two magnifying lenses in front of a pair of LG L3s or any other coming 2K phone display and you have a 2K per eye  WFOV HMD .. better than CV1  of the Rift, Project Morpheus or the Valve prototype.. however though I am sure  others have tested this .. all WFOV HMDs based on this FOV2GO design share one characteristic NAUSEA.. which is a clinical optical neurological phenomenon, which after 3 years of at least two companies developing HMDS independently along this path ..Sony and Facebook OVR .. we should be noting is a function of WFOV FOV2GO based HMDs ..

Further evidence: Recently Sony cancelled the Project Morpheus VR version of their fabled VR driving game .. because of NAUSEA

Thus we have irrefutable data/evidence from two independently WFOV FOV2GO derived HMD design teams producing  products that produce Nausea..

Mitigating nausea is not the solution .. ELIMINATING nausea is the only thing that will make this WFOV FOV2GO derived HMD design commercially viable .. 

Caveat :

I would say to the three OVR Amigos Abrash,Carmack and Luckey.. first ola seniors .. second congrats again on the Facebook buyout and third and most important that The Elder Scrolls Online IS the most PLEASANT Oculus Rift experience I have had in over 10 months of testing games with it .. the pace of the game, the gentler head tracking movements, the scale and beauty of the world , the soundscapes and atmosphere all of the things I note make it perfect for HMD gaming also apply to VR gaming ..and it's current shortcomings 

So I can live with the pain on the Rift for a game this good, sorry for an experience this good. 

It also happens to be the most requested game to play on the Rift .. 

With the first Beta over .. I set about uploading all this video .. and presently in my inbox came the invite for a second Beta.. with the client updated it was time to see if all of the above was just a wonderful delusion ... which if you know .. can be real sometimes..

The Elder Scrolls Online Beta 2  16/03/14, 17/03/14

Cinemizer OLED 

On the Cinemizer OLED (and the smd ST1080) we have the option of playing in 2D as well as 3D.. head tracking still works of course .. so if your hardware cannot handle 3D, or your wetware cannot handle 3D, or if you are playing on an Xbox One or PS4 .. you can still play TESO.. on these HMDs .. like so..

 

The Elder Scrolls Online Oculus Rift 1080p sbs TriDef 3D Zeiss Head Tracking Beta 2 part I,II ,III  

What follows are 3 sustained plays of TESO with the Rift Dev Kit 1: these are real gaming sessions, just 3 I could upload in the time I had.. but you can see we can play this game on Rift and for considerable amounts of time.. 

 

You may be totally useless at fighting or combat in MMOs .. at an early point in TESO you start  working things out in your own way and time  .. pretty soon you are pulling off combos, combining with fellow travelers to take on adversaries, experimenting and learning at your own pace ..as a result  far more skilled and satisfying than just mouse clicking or hitting the number keys ..  blocks, dodges counter attacks and exploits all start weaving their way into your combat..

Where other MMOs have you following or chasing or dragging a cursor, TESO has you focusing on looking at your enemy and looking for a weakness .. or exploit

And this makes playing it with head tracking on a HMD the perfect way to play.

TESO is the greatest experience you can have with your Oculus Rift Dev Kit 1 or 2 (of?) . After 10 or so months TVRKING the Rift, playing TESO with which is the best experience ( with a a Rift!) I have had .. or could recommend to anyone.

Take a trip into VR Tamriel and you will soon realize TESO is  THE most significant game for mass HMD and VR use to take off. 

So with all due respect to VR Creationists ... I believe in the augmented gamer .. evolving how we game and evolving VR and HMD use .. point in case I could have spent 4 minutes in a shark cage on PS4 with Project Morpheus or 4 minutes in a Crystal Cove space ship .. I just spent 27 hours in The Elder Scrolls Online .. in VR and 3D .. which do you think Mr Carmack , Luckey , Abrash and Yoshida san ..is going to be more popular with real world gamers outside controlled show demo environments and industry insider speeches? And which do you think is more relevant to the evolution of the consumer HMD and VR in general ..? 

The HMD developer that gives the users what they want ...listens to what they want to play as opposed to force feeding them game ideas rooted in late night highlighting sessions of 90s scifi .. will emerge victorious..content is king....  all it takes is Microsoft turning up with HALO VR or Nintendo with Mario and Zelda VR (Metroid VR?) and everything just gets reset .. mayhaps GabeN will gift OVR Half Life 3.. ?

Experiencing or living games like The Elder Scrolls Online with a world so carefully crafted and wholly crafted could take half a decade to arrive in pure VR form.. it took Rare 5 years to produce Goldeneye on an N64 .... the issue with pure VR gaming is as the hardware devs have started to admit .. the hardware, display and input tech has issues within issues and cannot  be finalized ..development is in a chaotic embryonic and Darwinian state of change,adaptation and evolution which the more astute of you will have gathered does not sit well with VR Creationists...who believe VR was created in a matter of days back in 2012 ..

What then for the VR or HMD fan?

It is a stunning waste of current technology not to allow non VR gaming on HMDs like the Rift and Project Morpheus... and there is no reason why you could not have been playing whatever you wanted to with your Rift over the last 10 months which is what I have been doing ..from day 1 with Arma III and Bioshock Infinite to  TESO .. and Titanfall .. 

Get me a PS4 Project Morpheus Dev Kit and I will be working out how to play Metal Gear Solid V, Killzone and Second Son on it ..whilst Sony decides whether to let the diver out of the shark cage.. or which game is suitable for VR .. or HMD use .. 

This is  the difference between a pilot or driver and the engineer that builds the hardware they use.. 

VR and HMD design has to move to a pilot/driver led evolution with the engineers sitting out the VR sermons ..or 5 years from now we will have excellent VR gaming engines with 0 public engagement .. as every one and their toddler scoots off to play Flappy Bird 12 instead once again consigning VR gaming to scifi..and that includes the developers that are so vital to kickstarting (in the less bitter taste leaving sense of the word) the so called VR gaming revolution.. 

I have much more to do with augmenting how we play TESO, but that is for another time, right which reminds me I have to go buy it now to play the full game .. 

See you in Tamriel .. and remember don't feed the VR sharks..the bite back's killer..

Epilogue and Thanks 

1. Bethesda thanks for the Invites and such an awesome game
2. Carl Zeiss 
3. siliconmicrodisplay
4. Oculus VR  
5. DDD and TriDef 3D .. stunning as always 

Find out more about the gear I use

Carl Zeiss Cinemizer OLED + Zeiss Head Tracker 
Siliconmicrodisplay ST1080
Oculus Rift 

this content is © and originated from thegameveda http://the-games-veda.blogspot.com/  if you are reading it from any other site that has not provided a link back it has been leached 

Preparing for VR : Darknet Oculus Rift's Killer Indie App played with LEAP Motion finger tracking

You cannot downplay the influence of 90s scifi on the HMD scene .. chief among these influences is of course is a most humble and self apologetic Mr Gibson .. E McNeill's excellent Darknet though influenced by the master .. does not hark back to that time 

A time when our economies were flush and people were so happy to have come out the other side of the Cold War without being turned into atomic shadows etched on what remained of their surroundings that they needed to listen to the bleakest of music from Grunge to Goth, over in Japan every other manga an anime was based on some failure of the perfect society, some angel leaping from a gilded cage, even the odd Terminator got depressed in the 90s .. 

This nihilistic vibe ran through the scifi of the era too and a subset of that was Cyberpunk .. gritty and wet and as dirty a vision of the future as was the industrial past it was based on .. 

Darknet although Cyberpunk .. is nothing like this .. as the author admits this is cyberpunk cleaned of all the grunge and grime not so much Cyber Punk as Clean Punk .. atm though there is only faintest inkling of a back story or plot..

.. this does not stop it from being one of most promising VR demos to date , impressive because of it's simplicity or minimalism and also because it already is very easy to combine LEAP Motion Finger Tracking with the Rift's head tracking ..

So this is the demo I have been lucky enough to play , this is very early stuff and there is a long way to go .. but like the 90s it's going to be a fantastic trip ..when the demo goes public next week

Eplilogue and thanks 

1. E McNeill many thanks for the demo 
2. Oculus VR Keep calm and carry on
3. Leap Motion .. stunning hardware that VR needs to be VR 


In order to use Leap Motion 

Install the Touchless App from Airspace 
Then start Darknet and assign forward and backward to the left mouse click ... 
Now in game all you need to do is point where you look and or move the on screen cursor with your hand or finger , poking forward completes the click .. and thus very simply you can combine head tracking with hand and finger tracking .. 

You will find many more examples of combining multiple VR leaning controls around my blog ... 

this content is © and originated from thegameveda http://the-games-veda.blogspot.com/  if you are reading it from any other site that has not provided a link back it has been leached 

Preparing for VR : Augmenting Infamous Second Son : A Second method to enable Zeiss head tracking in all PS4 games ..allowing combination of multiple VR leaning controls+optimizing and accelerating head tracking

Time to discuss a second method for enabling Zeiss head tracking in all PS4 and Xbox One games .. ( this works for all PS3 and Xbox 360 games too) .. The experience is latency, lag and drift free ( JC)  for the user .. in fact like the method I used for old gen consoles you will find you need to slow it down in some cases .. every tiny field of view movement you see on this page is controlled using Cinemizer OLED head tracking on the PlayStation 4...

This method also allows the combination of multiple VR leaning controls ...existing controls ..from Kinect to Leap Motion ..so you could if you were so inclined introduce Kinect control to the PS4.. 

We will be exploiting the principle of offloading which creates the possibility of accelerating, fine tuning and "tweaking" VR leaning control inputs ..  

I will be using this very simple method  to augment how I play Infamous Second Son, currently the finest game on PS4 and the most stunning next gen console game in existence.. (till the next one) 

Why Bother?

A Game of Thumbs..

Almost 40 years into gaming we still move that in game avatar and in this case his field of view camera at the speed of our thumbs .. with the talent of our thumbs .. or on PC ..mice.. 

That's not good enough for me, being spoilt by using head tracking in console games including Infamous and Infamous 2 for over a year now ...

Solution : we introduce Zeiss head tracking to control the FOV and let go of the thumbstick..on PlayStation 4.. now.. 

Let's illustrate this with a Michael Bay moment from Infamous Second Son ....achieved through Zeiss head tracking .. 

Second Son is a lightning fast game, the thumb stick is very crude way of moving the field of view for.. The Zeiss head tracker is far more precise, I never need to "paint" targets in the game in a battle, I am already looking at them .. and aiming .. fast accurate and user definable head tracking is far better at third person camera movement than a thumb stick which often in third person games on consoles at least has that annoying snap back to centre view action..if you let go ..and then you have to nudge it back ..  playing with head tracking is a revelation .. an augmented experience..it's how you always wanted to play.. or expected to play..

The speed of this game means playing on a HMD  requires not only a rock solid head tracker, one that will not go randomly off in the wrong direction at the right time, but also a head mounted display that is designed for rapid motion .. without nausea and without the need to exert force .. and right now the Cinemizer OLED is still it .. fitting, ergonomics, optics .. this is the form factor that all HMDs should aim for .. a pair of shades .. <180g and a very small silhouette .. anyone can slap a tablet screen into a 3D printed FOV2GO array .. how many can begin to pack that experience into a form factor as easy to use as a pair of shades?

Try playing a game like this with a WFOV HMD and you will see why Sony cancelled the Morpheus version of their fabled driving game .. this is a game that requires lightning fast aiming and targeting .. all of which this HMD is suited to .. take it up to a wider FOV and HMD size increases, bulk, momentum, nausea and optical issues all come into play .. each of these then revealing further issues .. of their own ... 

This is an example of where a LFOV HMD designed for rapid motion without nausea is called for .. if you thought all PS4 games were going VR after Morpheus arrives .. when it arrives .. you would be mistaken and sore (very) from using the device..

Putting and end to Stop and Swipe ...

Although Second Son like the Crack Down and Prototype series relies on fast flowing fluid game play, that fluidity is often broken by the understandable yet not exactly very suitable touch pad swiping that Sony probably insisted on developers including in the first gen of PS4 games, much like PS3 first gen games all had sixaxis rattle and shake control schemes.  

this is a stop and swipe moment .. the frequency of which is directly proportional to your intended Infamy.. 

Unlike Prototype or Crack Down where you never really need to stand still or take your hands off the joy pad ( note to self do not include touch pad or gyro controls that require the gamer to stand still, stop mid action and swipe a touch pad). In Second Son you will have to engage in this every 60s or so, once your "energy" bar runs down and you have to swipe to drain an energy source ...sometimes in the middle of a battle 

So there I am morphing into napalm and zooming across rooftops, taking out enemies left right and centre, saving the good grunge and coffee loving citizens of Seattle when .. I have to stop, stand next to a chimney, let go of the joy pad and what was that touch pad action..press both sides together ? swipe left or right ...or up? Yeah I can remember but why disrupt the flow of game to include mandatory abstract touch controls ?

You would think that people who spend 10s of millions designing controllers and hardware would actually try playing games with the things ... .. touch control though well suited to the likes of Flappy Bird .. is a pain for anyone trying to play third or first person games .. Android and iOS gamers often resort to PS3 joy pads to play .. so placing a touch pad on a joy pad may have been a cool engineering design achievement but it is a folly .. useless in every game genre.. take your hands off the steering wheel in racing games, brake your combos in fighting games , release the fov thumb stick to launch a drone in Killzone ..  or stand still in Infamous Second Son  :

this is an example where you stand still take your hands off the DS4 to swipe at the touch pad .. so not Michael Biehn..

We want to eliminate this abstract legacy control that belongs on a phone  and augment the in game action it is assigned to using the Zeiss head tracker  .. like so 

the slightest downward movement of the Zeiss head tracker now initiates the smoke drain eliminating the need to swipe at the
touch pad .. you literally look at the source of smoke to drain it.. 

The following videos shows a sustained play of the game using the Cinemizer OLED HMD with Cinemizer OLED head tracking using both methods available at this time.. 

Infamous Second Son played with Cinemizer OLED Head Tracking Method 1 : Xim 4E avec Beanie

Note the blue blur? That's me having to switch to a second PS4 joy pad to use the touch pad swipe to use the drain/recharge game mechanic .. 

I love the DS4 design, but devs need to offer gamers the option of switching off or avoiding touch controls .. both Killzone and Infamous suffer from their inclusion .. and are not in any way enhanced by their inclusion .. as for VR and Project Morpheus .. we shouldn't be using any legacy control at all.. 

In the above test Head Tracking is achieved on PS4 using the method I described here 

Preparing for VR : Enabling Cinemizer OLED Head Tracking in ALL PlayStation 4 games

If anything, touch, like motion is a legacy control ..in this context tacked onto a joy pad, a hindrance to the flow of gaming .. 

Whilst looking at a second way to implement Zeiss head tracking on next gen consoles I came across a very simple solution for  augmenting the legacy touch control out of  Infamous Second Son ...

When you watch my second play of Second Son using a second method for enabling head tracking in PS4 games... you should note all those blue blurs in the video and silly Beanie hat  above are gone:

Infamous Second Son played with Cinemizer OLED HMD and Head Tracking Method 2  :Controller Max +PC Bridge  sans Beanie..





To eliminate the problem of having to swipe at a DS4 touch pad I mapped the touch pad  function to a mouse downward movement..

How : 



you will need the following ...in addition to a PS4..salient details .. ) 

1. Controller Max/Cronus max/Titan One ( they are all the same thing !) with 1.55 firmware or above

CronusMax - Cronus Max for Xbox One/PS4/PS3/Xbox 360/Wii/PC 

2.Cinemizer OLED Head Tracker + HMD

3. A laptop/PC "bridge"  running GTUNER with the maxaim plugin installed

1. Connect your Controller Max/Titan One/Cronus Max to your PS4 via a decent USB extension cable. 
2. Connect the programming port of the device to a USB port on your laptop. 
3. Run GTuner and Maxaim .. select a PS4 blank FPS profile . 
4. Map the Right analogue to  mouse up, down, right and left .
5.  Map the touch pad PRESS DOWN FN to mouse down..  THIS ELIMINATES THE NEED TO USE THE TOUCH PAD TO RECHARGE/DRAIN POWER FROM SOURCES.. 
6. Connect the Zeiss head tracker to a USB port on your laptop .. and then connect a PS4 Dual Shock 4 to the Cronus device with a micro USB cable .  
7. Start your game .. set the look "speed" to fastest in the game options or one that suits the way you move .. your head ...
8. Now if you want to accelerate  the head tracking .. adjust mouse sensitivity .. to "decelerate" decrease mouse sensitivity .. you can fine tune this using Maxaim and GTuner , you could also adjust the sensitivity from Windows mouse settings .. either way you have better than 1: 1 head tracking , which importantly does not drift or lag .. you will find yourself having to slow it down .. 

You can also combine Leap Motion and or Kinect control to PS4 games this way

The use of a laptop bridge connected to your Controller max connected to your  PS4 means anything we connect to that laptop (within reason .. a banana will not work)  can be mapped to an in game action on the PlayStation 4 .. just like I was demoing Leap Motion with Kinect on Xbox 360 and Child of Eden with Leap Motion and head tracking on PS3 ( in 2D and 3D) for Mizuguchi san.. last year :

Preparing for VR: Combining LEAP Motion+Cinemizer OLED headtracking+Kinect Voice Recog in PS3 and Xbox 360 games

At the moment there is a lack of games to test on next gen consoles, but using the method I described here you can introduce Leap Motion and Kinect and combine them with Xbox One and PS4 games .. or go back and play all your Xbox 360 and PS3 favourites ..including Dark Souls 2 .. more on that another time..

Right now there is nothing stopping you from using Zeiss head tracking to play any old gen and next gen console game you want , 2D and 3D .. I am sure you can think of a few you would like to try out .. 

Enabling Zeiss head tracking in ALL PS4 games method 3 : 

The ultimate way to enable head tracking in console games now and include legacy games .. i.e. non VR .. is for developers to enable mouse control in their games .. The PS4 already has a few games that allow this .. 

For Project Morpheus to take off , it must allow non VR legacy gaming and the simplest way to do this is enable mouse control in PS4 games ..even if the on board head tracker does not do mouse emulation we can always use Zeiss head tracking with the PS4 and Project Morpheus .. as it detaches from the Cinemizer HMD .. 

But what does it all mean for head tracker implementation  for HMD developers?

I have been testing and using head tracking on HMDS, PCs and consoles now for 16 months, tested it on thousands from E3 to Eurogamer last year.. including my own obsessive testing ..I have more hours with head tracking across more devices and HMDs than most people working back at OVR or other HMD manufacturers.. 

Dispelling The Myth of VR Head Tracker Lag Latency and Drift..Again .. 

As we can see from the above Zeiss Head Tracking (combined with displays),  I have used in HMDs now since 2012 .. does NOT suffer from perceptible latency, lag or drift issues .. 


If you are in a lab trying to figure out whose head tracker sensor is faster .. hold on .. head tracking in vivo .. is far different to in vitro 

The assumption that all head tracking , like the assumption that all HMD and VR use .. involves issues .. like drift and drag, or nausea is quite obviously flawed .. 

I have 16 months and 1000s of witnesses , to the fact ..

So again Mr Carmack and this time respectfully (as I want my dev kit 2 asap!) I have to disagree with your tweet back to me .. 

There simply is no lag perceptible using Zeiss head tracking on PC , old gen and next gen consoles..

Nausea and lag are functions of SOME HMDs.. WFOV HMDS .. like the Rift and Project Morpheus.. it matters not if they are reduced .. what matters is they are not perceptible to the end user .. And they are a function of the hardware and HMD designs some manufacturers are using.. 

The solution to head tracker Lag at least, is not just faster sensors .. if you are embedding those sensors on a circuit board mounted to the display itself , in a form factor like current WFOV HMDs .. you are to all intents and purposes attaching a butterfly to a brick ..  it's the object of larger mass the brick/Rift that obviously negates the grace of the of butterfly/head tracker.. and this brick is then ski mask attached to your head .. is that obvious enough?  

The weight, bulk, size, ergonomics and placement of the head tracker all play a part .. as well as momentum .. but I have gone on about this way back and $2 billion later you guys really should have figured this out by now .. I just had a vision of OVR interns being sent out to trap butterflies..  

This holds true for PC and console gaming.

There are now at least two forms of established non optical based HMD head tracking : mouse emulation and DOF axes  : Neither of these need suffer from perceptible lag or drift .. and both can and have been combined in one product..

Let me say that again : Mouse emulation head tracking does NOT suffer from drift in the manner experienced by Palmer way back in 2012 .. hopefully you have experienced Zeiss head tracking since and can see this .. and this is on 2012 hardware.. 

Whilst Mouse emulation does miss the role .. no gamer actually misses a role function when they are playing Halo, Titan Fall , Minecraft, The Elder Scrolls Online or  as I played  Sucker Punch's  Infamous Second Son..  

Across the full spectrum of tests I did on  HMD usage  ..from 7 year old Minecraft kids running their own servers to 40 somethings hungry for VR .. promised to them in the 80s and 90s .. no one could notice the difference or absence of a role function .. what mattered was they moved their head .. that moved their FOV in game .. without any perceptible latency .. without random drift and without nausea.. I had kids crying that they had to leave  their sessions of HMD Minecraft.. 

One day in the future , there may be a real world demonstrable and more importantly perceptible benefit of the added role axis .. right now though you can approximate head tracking augment any game with it with an efficacy that still places the end user in the game .. 

Mouse Emulation head tracking is at a more advanced state than DOF axes head tracking at this stage : Having existed in  the Zeiss head tracker since 2012 and available as a retail product since 2013.. used in gaming and serious applications.. 

DOF axes head tracking : still suffers from drift and lag still ( just try your Rift now .. as an example when trying out Dark Net on DK1 I can feel the lag in the head tracker )  

Evolving and Optimising  Head Tracker Design and code

DOF axes head tracking on Rift suffers from more issues than mouse emulation head tracking using a Zeiss head tracker with the same Rift .. 

What are the solutions and how can head tracking be optimized or accelerated ?

• Dynamic Switching of or combination of mouse head tracking and DOF axes :could this be a solution to Oculus Rift head tracker lag and drift and further accelerating Zeiss head tracking?

The Zeiss head tracker combines both mouse emulation and DOF axes ,  on the Rift both of these seem to be heavily reliant on your PC hardware doing the grunt work  and correction .. on the Zeiss far more of this process happens in hardware .. the head tracker hardware . And the firmware and code and all those lovely quaternion equations are rock solid ..  now what if we could combine and exploit both .. to minimise CPU load .. using mouse emulation for CPU intensive tasks and switching to full DOF axes when a role is necessary ..and then back again .. in a way that the user does not perceive . .

This is something I am going to test with the Zeiss  when I have time ..  

• Offloading and possibly accelerating VR input calculations 

In the very basic, technically simple and cost effective method I am using here to enable head tracking on the PS4 :

We can accelerate VR leaning control inputs as well as take the load off the console or PC 

All of this for as little as $59.99 .. or at no cost at all when console developers start to include support for mouse control and or DOF axes..

Both the XIM and the Controller Max devices seem to accelerate the head tracker input , taking the load off the console CPU .. allowing for lag and drift free head tracking with the Zeiss head tracker even on Xbox 360 and PS3 games.. 

The Rift head tracker does not do mouse emulation in hardware so cannot be used with devices like the XIM or Controller Max, mouse emulation on the Rift is still at an embryonic stage that the awesome TriDef folks are working on.. However it will always be software side .. and that is your layer of lag.. 

That so little is known by so many about the real world usage of Head Mounted Displays and VR leaning controls like head tracking is what fascinates me about this new frontier .. 

On the other hand the old boys that design console hardware really should know better than to put touch pads on to joy pads ... 

Still if you follow the method above you can begin to augment out those pesky touch controls .. and play the game how you want to play.. as an augmented gamer...


Epilogue and thanks 

1. Sucker Punch ..awesome game 
2. Controller Max /Titan One.. what's in a name?
3. Carl Zeiss 

this content is © and originated from thegameveda http://the-games-veda.blogspot.com/  if you are reading it from any other site that has not provided a link back it has been leached