Preparing for VR : Augmenting Infamous Second Son : A Second method to enable Zeiss head tracking in all PS4 games ..allowing combination of multiple VR leaning controls+optimizing and accelerating head tracking

Time to discuss a second method for enabling Zeiss head tracking in all PS4 and Xbox One games .. ( this works for all PS3 and Xbox 360 games too) .. The experience is latency, lag and drift free ( JC)  for the user .. in fact like the method I used for old gen consoles you will find you need to slow it down in some cases .. every tiny field of view movement you see on this page is controlled using Cinemizer OLED head tracking on the PlayStation 4...

This method also allows the combination of multiple VR leaning controls ...existing controls ..from Kinect to Leap Motion ..so you could if you were so inclined introduce Kinect control to the PS4.. 

We will be exploiting the principle of offloading which creates the possibility of accelerating, fine tuning and "tweaking" VR leaning control inputs ..  

I will be using this very simple method  to augment how I play Infamous Second Son, currently the finest game on PS4 and the most stunning next gen console game in existence.. (till the next one) 

Why Bother?

A Game of Thumbs..

Almost 40 years into gaming we still move that in game avatar and in this case his field of view camera at the speed of our thumbs .. with the talent of our thumbs .. or on PC ..mice.. 

That's not good enough for me, being spoilt by using head tracking in console games including Infamous and Infamous 2 for over a year now ...

Solution : we introduce Zeiss head tracking to control the FOV and let go of the thumbstick..on PlayStation 4.. now.. 

Let's illustrate this with a Michael Bay moment from Infamous Second Son ....achieved through Zeiss head tracking .. 

Second Son is a lightning fast game, the thumb stick is very crude way of moving the field of view for.. The Zeiss head tracker is far more precise, I never need to "paint" targets in the game in a battle, I am already looking at them .. and aiming .. fast accurate and user definable head tracking is far better at third person camera movement than a thumb stick which often in third person games on consoles at least has that annoying snap back to centre view action..if you let go ..and then you have to nudge it back ..  playing with head tracking is a revelation .. an augmented experience..it's how you always wanted to play.. or expected to play..

The speed of this game means playing on a HMD  requires not only a rock solid head tracker, one that will not go randomly off in the wrong direction at the right time, but also a head mounted display that is designed for rapid motion .. without nausea and without the need to exert force .. and right now the Cinemizer OLED is still it .. fitting, ergonomics, optics .. this is the form factor that all HMDs should aim for .. a pair of shades .. <180g and a very small silhouette .. anyone can slap a tablet screen into a 3D printed FOV2GO array .. how many can begin to pack that experience into a form factor as easy to use as a pair of shades?

Try playing a game like this with a WFOV HMD and you will see why Sony cancelled the Morpheus version of their fabled driving game .. this is a game that requires lightning fast aiming and targeting .. all of which this HMD is suited to .. take it up to a wider FOV and HMD size increases, bulk, momentum, nausea and optical issues all come into play .. each of these then revealing further issues .. of their own ... 

This is an example of where a LFOV HMD designed for rapid motion without nausea is called for .. if you thought all PS4 games were going VR after Morpheus arrives .. when it arrives .. you would be mistaken and sore (very) from using the device..

Putting and end to Stop and Swipe ...

Although Second Son like the Crack Down and Prototype series relies on fast flowing fluid game play, that fluidity is often broken by the understandable yet not exactly very suitable touch pad swiping that Sony probably insisted on developers including in the first gen of PS4 games, much like PS3 first gen games all had sixaxis rattle and shake control schemes.  

this is a stop and swipe moment .. the frequency of which is directly proportional to your intended Infamy.. 

Unlike Prototype or Crack Down where you never really need to stand still or take your hands off the joy pad ( note to self do not include touch pad or gyro controls that require the gamer to stand still, stop mid action and swipe a touch pad). In Second Son you will have to engage in this every 60s or so, once your "energy" bar runs down and you have to swipe to drain an energy source ...sometimes in the middle of a battle 

So there I am morphing into napalm and zooming across rooftops, taking out enemies left right and centre, saving the good grunge and coffee loving citizens of Seattle when .. I have to stop, stand next to a chimney, let go of the joy pad and what was that touch pad action..press both sides together ? swipe left or right ...or up? Yeah I can remember but why disrupt the flow of game to include mandatory abstract touch controls ?

You would think that people who spend 10s of millions designing controllers and hardware would actually try playing games with the things ... .. touch control though well suited to the likes of Flappy Bird .. is a pain for anyone trying to play third or first person games .. Android and iOS gamers often resort to PS3 joy pads to play .. so placing a touch pad on a joy pad may have been a cool engineering design achievement but it is a folly .. useless in every game genre.. take your hands off the steering wheel in racing games, brake your combos in fighting games , release the fov thumb stick to launch a drone in Killzone ..  or stand still in Infamous Second Son  :

this is an example where you stand still take your hands off the DS4 to swipe at the touch pad .. so not Michael Biehn..

We want to eliminate this abstract legacy control that belongs on a phone  and augment the in game action it is assigned to using the Zeiss head tracker  .. like so 

the slightest downward movement of the Zeiss head tracker now initiates the smoke drain eliminating the need to swipe at the
touch pad .. you literally look at the source of smoke to drain it.. 

The following videos shows a sustained play of the game using the Cinemizer OLED HMD with Cinemizer OLED head tracking using both methods available at this time.. 

Infamous Second Son played with Cinemizer OLED Head Tracking Method 1 : Xim 4E avec Beanie

Note the blue blur? That's me having to switch to a second PS4 joy pad to use the touch pad swipe to use the drain/recharge game mechanic .. 

I love the DS4 design, but devs need to offer gamers the option of switching off or avoiding touch controls .. both Killzone and Infamous suffer from their inclusion .. and are not in any way enhanced by their inclusion .. as for VR and Project Morpheus .. we shouldn't be using any legacy control at all.. 

In the above test Head Tracking is achieved on PS4 using the method I described here 

Preparing for VR : Enabling Cinemizer OLED Head Tracking in ALL PlayStation 4 games

If anything, touch, like motion is a legacy control ..in this context tacked onto a joy pad, a hindrance to the flow of gaming .. 

Whilst looking at a second way to implement Zeiss head tracking on next gen consoles I came across a very simple solution for  augmenting the legacy touch control out of  Infamous Second Son ...

When you watch my second play of Second Son using a second method for enabling head tracking in PS4 games... you should note all those blue blurs in the video and silly Beanie hat  above are gone:

Infamous Second Son played with Cinemizer OLED HMD and Head Tracking Method 2  :Controller Max +PC Bridge  sans Beanie..





To eliminate the problem of having to swipe at a DS4 touch pad I mapped the touch pad  function to a mouse downward movement..

How : 



you will need the following ...in addition to a PS4..salient details .. ) 

1. Controller Max/Cronus max/Titan One ( they are all the same thing !) with 1.55 firmware or above

CronusMax - Cronus Max for Xbox One/PS4/PS3/Xbox 360/Wii/PC 

2.Cinemizer OLED Head Tracker + HMD

3. A laptop/PC "bridge"  running GTUNER with the maxaim plugin installed

1. Connect your Controller Max/Titan One/Cronus Max to your PS4 via a decent USB extension cable. 
2. Connect the programming port of the device to a USB port on your laptop. 
3. Run GTuner and Maxaim .. select a PS4 blank FPS profile . 
4. Map the Right analogue to  mouse up, down, right and left .
5.  Map the touch pad PRESS DOWN FN to mouse down..  THIS ELIMINATES THE NEED TO USE THE TOUCH PAD TO RECHARGE/DRAIN POWER FROM SOURCES.. 
6. Connect the Zeiss head tracker to a USB port on your laptop .. and then connect a PS4 Dual Shock 4 to the Cronus device with a micro USB cable .  
7. Start your game .. set the look "speed" to fastest in the game options or one that suits the way you move .. your head ...
8. Now if you want to accelerate  the head tracking .. adjust mouse sensitivity .. to "decelerate" decrease mouse sensitivity .. you can fine tune this using Maxaim and GTuner , you could also adjust the sensitivity from Windows mouse settings .. either way you have better than 1: 1 head tracking , which importantly does not drift or lag .. you will find yourself having to slow it down .. 

You can also combine Leap Motion and or Kinect control to PS4 games this way

The use of a laptop bridge connected to your Controller max connected to your  PS4 means anything we connect to that laptop (within reason .. a banana will not work)  can be mapped to an in game action on the PlayStation 4 .. just like I was demoing Leap Motion with Kinect on Xbox 360 and Child of Eden with Leap Motion and head tracking on PS3 ( in 2D and 3D) for Mizuguchi san.. last year :

Preparing for VR: Combining LEAP Motion+Cinemizer OLED headtracking+Kinect Voice Recog in PS3 and Xbox 360 games

At the moment there is a lack of games to test on next gen consoles, but using the method I described here you can introduce Leap Motion and Kinect and combine them with Xbox One and PS4 games .. or go back and play all your Xbox 360 and PS3 favourites ..including Dark Souls 2 .. more on that another time..

Right now there is nothing stopping you from using Zeiss head tracking to play any old gen and next gen console game you want , 2D and 3D .. I am sure you can think of a few you would like to try out .. 

Enabling Zeiss head tracking in ALL PS4 games method 3 : 

The ultimate way to enable head tracking in console games now and include legacy games .. i.e. non VR .. is for developers to enable mouse control in their games .. The PS4 already has a few games that allow this .. 

For Project Morpheus to take off , it must allow non VR legacy gaming and the simplest way to do this is enable mouse control in PS4 games ..even if the on board head tracker does not do mouse emulation we can always use Zeiss head tracking with the PS4 and Project Morpheus .. as it detaches from the Cinemizer HMD .. 

But what does it all mean for head tracker implementation  for HMD developers?

I have been testing and using head tracking on HMDS, PCs and consoles now for 16 months, tested it on thousands from E3 to Eurogamer last year.. including my own obsessive testing ..I have more hours with head tracking across more devices and HMDs than most people working back at OVR or other HMD manufacturers.. 

Dispelling The Myth of VR Head Tracker Lag Latency and Drift..Again .. 

As we can see from the above Zeiss Head Tracking (combined with displays),  I have used in HMDs now since 2012 .. does NOT suffer from perceptible latency, lag or drift issues .. 


If you are in a lab trying to figure out whose head tracker sensor is faster .. hold on .. head tracking in vivo .. is far different to in vitro 

The assumption that all head tracking , like the assumption that all HMD and VR use .. involves issues .. like drift and drag, or nausea is quite obviously flawed .. 

I have 16 months and 1000s of witnesses , to the fact ..

So again Mr Carmack and this time respectfully (as I want my dev kit 2 asap!) I have to disagree with your tweet back to me .. 

There simply is no lag perceptible using Zeiss head tracking on PC , old gen and next gen consoles..

Nausea and lag are functions of SOME HMDs.. WFOV HMDS .. like the Rift and Project Morpheus.. it matters not if they are reduced .. what matters is they are not perceptible to the end user .. And they are a function of the hardware and HMD designs some manufacturers are using.. 

The solution to head tracker Lag at least, is not just faster sensors .. if you are embedding those sensors on a circuit board mounted to the display itself , in a form factor like current WFOV HMDs .. you are to all intents and purposes attaching a butterfly to a brick ..  it's the object of larger mass the brick/Rift that obviously negates the grace of the of butterfly/head tracker.. and this brick is then ski mask attached to your head .. is that obvious enough?  

The weight, bulk, size, ergonomics and placement of the head tracker all play a part .. as well as momentum .. but I have gone on about this way back and $2 billion later you guys really should have figured this out by now .. I just had a vision of OVR interns being sent out to trap butterflies..  

This holds true for PC and console gaming.

There are now at least two forms of established non optical based HMD head tracking : mouse emulation and DOF axes  : Neither of these need suffer from perceptible lag or drift .. and both can and have been combined in one product..

Let me say that again : Mouse emulation head tracking does NOT suffer from drift in the manner experienced by Palmer way back in 2012 .. hopefully you have experienced Zeiss head tracking since and can see this .. and this is on 2012 hardware.. 

Whilst Mouse emulation does miss the role .. no gamer actually misses a role function when they are playing Halo, Titan Fall , Minecraft, The Elder Scrolls Online or  as I played  Sucker Punch's  Infamous Second Son..  

Across the full spectrum of tests I did on  HMD usage  ..from 7 year old Minecraft kids running their own servers to 40 somethings hungry for VR .. promised to them in the 80s and 90s .. no one could notice the difference or absence of a role function .. what mattered was they moved their head .. that moved their FOV in game .. without any perceptible latency .. without random drift and without nausea.. I had kids crying that they had to leave  their sessions of HMD Minecraft.. 

One day in the future , there may be a real world demonstrable and more importantly perceptible benefit of the added role axis .. right now though you can approximate head tracking augment any game with it with an efficacy that still places the end user in the game .. 

Mouse Emulation head tracking is at a more advanced state than DOF axes head tracking at this stage : Having existed in  the Zeiss head tracker since 2012 and available as a retail product since 2013.. used in gaming and serious applications.. 

DOF axes head tracking : still suffers from drift and lag still ( just try your Rift now .. as an example when trying out Dark Net on DK1 I can feel the lag in the head tracker )  

Evolving and Optimising  Head Tracker Design and code

DOF axes head tracking on Rift suffers from more issues than mouse emulation head tracking using a Zeiss head tracker with the same Rift .. 

What are the solutions and how can head tracking be optimized or accelerated ?

• Dynamic Switching of or combination of mouse head tracking and DOF axes :could this be a solution to Oculus Rift head tracker lag and drift and further accelerating Zeiss head tracking?

The Zeiss head tracker combines both mouse emulation and DOF axes ,  on the Rift both of these seem to be heavily reliant on your PC hardware doing the grunt work  and correction .. on the Zeiss far more of this process happens in hardware .. the head tracker hardware . And the firmware and code and all those lovely quaternion equations are rock solid ..  now what if we could combine and exploit both .. to minimise CPU load .. using mouse emulation for CPU intensive tasks and switching to full DOF axes when a role is necessary ..and then back again .. in a way that the user does not perceive . .

This is something I am going to test with the Zeiss  when I have time ..  

• Offloading and possibly accelerating VR input calculations 

In the very basic, technically simple and cost effective method I am using here to enable head tracking on the PS4 :

We can accelerate VR leaning control inputs as well as take the load off the console or PC 

All of this for as little as $59.99 .. or at no cost at all when console developers start to include support for mouse control and or DOF axes..

Both the XIM and the Controller Max devices seem to accelerate the head tracker input , taking the load off the console CPU .. allowing for lag and drift free head tracking with the Zeiss head tracker even on Xbox 360 and PS3 games.. 

The Rift head tracker does not do mouse emulation in hardware so cannot be used with devices like the XIM or Controller Max, mouse emulation on the Rift is still at an embryonic stage that the awesome TriDef folks are working on.. However it will always be software side .. and that is your layer of lag.. 

That so little is known by so many about the real world usage of Head Mounted Displays and VR leaning controls like head tracking is what fascinates me about this new frontier .. 

On the other hand the old boys that design console hardware really should know better than to put touch pads on to joy pads ... 

Still if you follow the method above you can begin to augment out those pesky touch controls .. and play the game how you want to play.. as an augmented gamer...


Epilogue and thanks 

1. Sucker Punch ..awesome game 
2. Controller Max /Titan One.. what's in a name?
3. Carl Zeiss 

this content is © and originated from thegameveda http://the-games-veda.blogspot.com/  if you are reading it from any other site that has not provided a link back it has been leached